-- Logs begin at Mon 2022-07-11 10:33:25 UTC, end at Mon 2022-07-11 12:13:02 UTC. -- Jul 11 11:04:18.777990 dv-cinder-848429-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 11 11:04:18.786354 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 11 11:04:18 2022] *** Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: nodename: dv-cinder-848429-2-iscsi Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: machine: x86_64 Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: clock source: unix Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: pcre jit disabled Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: detected number of CPU cores: 4 Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: current working directory: / Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: detected binary path: /usr/bin/uwsgi-core Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: your processes number limit is 47680 Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: your memory page size is 4096 bytes Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: detected max file descriptor number: 2048 Jul 11 11:04:18.789539 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: lock engine: pthread robust mutexes Jul 11 11:04:18.798469 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: thunder lock: enabled Jul 11 11:04:18.798469 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 11 11:04:18.798469 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 11 11:04:18.834115 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: Python main interpreter initialized at 0x55ba22c6a9c0 Jul 11 11:04:18.834115 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: python threads support enabled Jul 11 11:04:18.834115 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: your server socket listen backlog is limited to 100 connections Jul 11 11:04:18.834115 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: your mercy for graceful operations on workers is 90 seconds Jul 11 11:04:18.834443 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: mapped 403077 bytes (393 KB) for 2 cores Jul 11 11:04:18.834547 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: *** Operational MODE: preforking *** Jul 11 11:04:18.834814 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: *** uWSGI is running in multiple interpreter mode *** Jul 11 11:04:18.834814 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: spawned uWSGI master process (pid: 80491) Jul 11 11:04:18.835017 dv-cinder-848429-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 11 11:04:18.836114 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: spawned uWSGI worker 1 (pid: 80499, cores: 1) Jul 11 11:04:18.836114 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: spawned uWSGI worker 2 (pid: 80500, cores: 1) Jul 11 11:04:18.836114 dv-cinder-848429-2-iscsi devstack@keystone.service[80491]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 11 11:04:24.296927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 11:04:24.296927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: from cryptography.utils import int_from_bytes Jul 11 11:04:24.297735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 11:04:24.297735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: from cryptography.utils import int_from_bytes Jul 11 11:04:24.298155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 11:04:24.298155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: from cryptography.utils import int_from_bytes Jul 11 11:04:24.298266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 11:04:24.298266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: from cryptography.utils import int_from_bytes Jul 11 11:04:24.346895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 11:04:24.347026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 11:04:24.347112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 11:04:24.347202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 11:04:24.347317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] command line args: [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 11:04:24.347406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] command line args: [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 11:04:24.347666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 11:04:24.347752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 11:04:24.347831 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ================================================================================ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 11:04:24.347910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ================================================================================ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 11:04:24.348088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] admin_token = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.348180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] admin_token = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.348330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] config_dir = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.348417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] config_dir = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.348501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.348636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.348722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] config_source = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.348846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] debug = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.348933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] config_source = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.349042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.349347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] debug = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.349347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.349577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.349676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.349777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.349886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.349988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_config_append = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_config_append = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_dir = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.350918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_file = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_dir = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.351337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.351422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_file = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.351542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.351724 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.351849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.352958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.353039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.353246 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.353384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.353555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.353657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.353793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.353985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.354079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.354231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.354395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.354487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.354564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.354711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.354803 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.354991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.355088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.355351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.355719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.355904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] publish_errors = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.355997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.356074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] publish_errors = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.356148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.356349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.356443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.356592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.356678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.356752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.356829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] standard_threads = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] standard_threads = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.357961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_journal = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_json = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_journal = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_stderr = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_json = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] use_syslog = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_stderr = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.358792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] use_syslog = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.359068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.359201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 11:04:24.359340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.359430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.359510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.359584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.359662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.359736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.359807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.359976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.360057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.360223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.360314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.360461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.360546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.360695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.360780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.360928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.361973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.362102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.362226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.362343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.362459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.362575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.362697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.362825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.362942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.363064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.363185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.363423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.363557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.363557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.363876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.363993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.364108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.364108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.364330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.364458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.364577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.364698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.364815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.364936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.365995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.366217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.366217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.366449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.366572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.366688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.366804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.366924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.367939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.368935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.369059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.369190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.369315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.369447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.369573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.369691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.369813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.369938 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.370064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.370178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.370294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.370411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.370528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.370650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.370765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.370881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.371963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.372080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.372204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.372322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.372322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.372547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.372671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.372795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.372795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.373941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.374065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.374191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.374317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.374444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.374562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.374685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.374810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.374928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.375051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.375242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.375378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.375496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.375612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.375729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.375859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.375978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.376095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.376212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.376327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.376443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.376573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.376703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.376832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.376957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.377087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.377244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.377362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.377478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.377604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.377722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.377839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.377957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.378073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.378189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.378320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.378439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.378555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.378672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.378790 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.378896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.379992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.380114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.380232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.380348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.380477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.380595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.380718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.380836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.380958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.381078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.381206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.381325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.381441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.381559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.381676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.381792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.381918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.382044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.382163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.382283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.382401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.382524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.382650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.382774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.382892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.383009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.383126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.383276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.383397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.383514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.383641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.383760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.383876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.external = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.external = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.384980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.385100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.385218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.password = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.385336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.password = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.385452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] auth.token = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.385570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] auth.token = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.385695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.385814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.385931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.386890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.387119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.387119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.387367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.387495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.387620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.387739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.387869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.387989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.388108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.388225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.388342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.388461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.388586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.388586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.388816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.388816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.389049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.389172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.389366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.389366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.389599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.389599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.389820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.389937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.390067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.390186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.390305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.390906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.391029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.391029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.391309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.391309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.391636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.391636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.391857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392619 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.392988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.393107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.393225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.393343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.393461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.393589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.393710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.393828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.393828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.394060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.394179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.394305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.394425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.394542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.394661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.394778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.394904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.395024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.395141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.395273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.395393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.395512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.395642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.395763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.395882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.396956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.397082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.397202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.397319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.397437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.397555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.397672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.397799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.397939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.398059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.398177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.398295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.398413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.398539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.398659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.398778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.398896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.399015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.399137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.399317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.399439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.399557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.399675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.399792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.399911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.400041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.400161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.400278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.400396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.400513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.400669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.400798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.400918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.401037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.401154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.401272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.401389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.401515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.401634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.401781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.401900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.402018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.402135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.402262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.402381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.402499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.402616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.402762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.402892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.403023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.403143 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.403339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.403457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.403574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.403692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.403864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.403987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.404107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.404225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.404342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.404460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.404586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.404706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.404824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.404941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.405059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.405178 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.405304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.405423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.405542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.405660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.405778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.405896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.406022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.406735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.406865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.406991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.407112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.407232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.407394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.407514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.407632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.407750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.407867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.407994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.408113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.408231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.408350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.408468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.408585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.408726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.408847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.408965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.409083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.409204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.409323 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.409450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.409569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.409687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.409805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.409923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.410041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.410168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.410287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.410407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.410525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.410643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.410761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.410887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.411989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.412107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.412225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.412351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.412471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.412589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.412707 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.412824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.412942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.413068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.413187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.413305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.413423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.413542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.413659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.413847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.413968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.414087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.414205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.414324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.414442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.414569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.414690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.414807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.414925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.415043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.415161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.415308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.415429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.415548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.415674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.415794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.415912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.416038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.416164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.416283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.416401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.416520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.416638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.416765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.416885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.417956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.418923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.419145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.419313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] role.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.419433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.419433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] role.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.419666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] role.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.419666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.419889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] role.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.419889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.420109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.420226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.420353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.420472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.420591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.420708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.420826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.420826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.421058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.421175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.421293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.421410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.421528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.421654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.421811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.422006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.422182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.422338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.422462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.422590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.422711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.422829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.422947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.423066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.423184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.423184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.423440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.423440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.423661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.423778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.423896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.424974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.425100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.425219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.425337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.425462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.425581 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.425698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.425816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.425933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.426058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.426179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.426544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.426665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.426782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.426906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.427033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.427153 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.427289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.427409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.427526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.427526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.427767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.427887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.428955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.429072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.429201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.429323 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.429441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.429558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.429675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.429792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.429917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.430992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.431109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.431226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.431418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.431540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.431658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.431658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.431876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.431876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.432103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.432221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.432338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.432457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.432574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.432700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.432836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.432964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.433083 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.433083 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.433382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.433502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.433620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.433738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.433855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.433981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.434100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.434217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.434334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.434452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.434570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.434570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.434804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.434804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.435940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.436171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.436171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.436392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.436392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.436612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.436612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.436842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.436960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 11:04:24.437079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 11:04:24.437079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 11:04:24.463878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.464106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.464404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.464499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.464668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.464889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.472809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.473454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.500942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.501164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.501462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.501693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.644274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.644452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.693449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.693682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.694299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.694533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.764505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.764693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.764839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.764948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.765030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.765106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.765180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.765301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.765388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.765539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.765627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.765828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.791701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.791928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792250 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.792892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.793021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.793114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.793237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.805071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.805176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.817969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818703 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.818906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.819034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.819117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.819243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.819365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 11:04:24.853561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.853766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 11:04:24.854029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.854126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.854241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 11:04:24.854326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 11:04:24.854537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.854620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.854768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 11:04:24.854860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.855067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.855225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.855326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.855401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.855544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.855793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.866123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.866394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.866655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.866739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.866927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.867012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.867189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.867487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.875287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.875793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.878513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.878790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.879043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.879128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.879344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.879443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.879619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.879722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.879871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.879954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.880138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.880223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.880404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.880487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.880666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.880747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.880931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.881014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.881190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.881273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.881456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.881725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.908434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.909090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.909980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.910255 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.910693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.910973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.915502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.916165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.916247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.916523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.916913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.923432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.923710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.924200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.924480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.928399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.928681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.928955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.929186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.929283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.929473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.929557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.929740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.929822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.930004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.930096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.930269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.930353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.930535 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.930799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.931063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.948792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.949066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.949570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.949844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.954145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.954927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.957765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.958054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.958549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.958842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.963143 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.963972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.966753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.967583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.970440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.970732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.971025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.971280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.971379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.971585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.971669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.971857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.971941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.972149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.972424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.972695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.991214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.991514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.991777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.992035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.992118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:24.992385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.992648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:24.992904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.000468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.001325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.002169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.002442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.003038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.003334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.005705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.005980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.006256 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.006482 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.006567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.006758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.006842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.007037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.007341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.007636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.019988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.020262 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.020535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.020810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.020894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.021082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.021167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.021392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.021672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.021951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.022580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.022878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.023500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.023801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.040611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.041557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.043878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.044848 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.047696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.048681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.049426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.050422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.051066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.051619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.051902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.052351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.052632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.052910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.059140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.060267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.060752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.062031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.064512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.064801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.065189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.065464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.065834 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.066129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.066522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.066807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.074507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.074788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.075063 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.075916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.076202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.076485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.082864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.083140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.083438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.083712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.083978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.084325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 11:04:25.084410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.084604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.084687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.084878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.084960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.085149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.085233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.085410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.085492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.085682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.085773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.085939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.086050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.086207 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.086318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.086479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.086596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.086869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.087143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.087432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.087697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.087954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 11:04:25.124311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ba22c6a9c0 pid: 80499 (default app) Jul 11 11:04:25.126513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ba22c6a9c0 pid: 80500 (default app) Jul 11 11:04:25.128253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d2b0a-4ac0-4464-85e6-e44f04069001 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:25.128544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d2b0a-4ac0-4464-85e6-e44f04069001 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:25.128821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d2b0a-4ac0-4464-85e6-e44f04069001 None None] PATH_INFO: `/v3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:25.132970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Mon Jul 11 11:04:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:33.933719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6eb6-9667-4957-a4cd-0507665b72ba None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:33.934261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6eb6-9667-4957-a4cd-0507665b72ba None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:33.934261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c6eb6-9667-4957-a4cd-0507665b72ba None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:33.939241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:33.944564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:33.944982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:33.945111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:34.013062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: INFO dbcounter [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] Registered counter for database keystone Jul 11 11:04:34.036255 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG oslo_db.sqlalchemy.engines [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80499) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 11:04:34.042455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writer thread running {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 11:04:34.216290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG passlib.handlers.bcrypt [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80499) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 11 11:04:34.216628 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG passlib.handlers.bcrypt [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80499) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 11 11:04:34.270448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:34.272059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:34.391185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 11 11:04:34.466775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c6eb6-9667-4957-a4cd-0507665b72ba None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:34.536587 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: INFO dbcounter [None req-566c6eb6-9667-4957-a4cd-0507665b72ba None None] Registered counter for database keystone Jul 11 11:04:34.556447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG oslo_db.sqlalchemy.engines [None req-566c6eb6-9667-4957-a4cd-0507665b72ba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80500) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 11:04:34.562624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writer thread running {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 11:04:34.681659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c6eb6-9667-4957-a4cd-0507665b72ba None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:34.685984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96652400-e204-4c5c-b6f4-074fc96f555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:34.686646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652400-e204-4c5c-b6f4-074fc96f555f admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:34.686965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652400-e204-4c5c-b6f4-074fc96f555f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:34.687263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96652400-e204-4c5c-b6f4-074fc96f555f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:34.695428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96652400-e204-4c5c-b6f4-074fc96f555f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:34.753020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WARNING keystone.common.rbac_enforcer.enforcer [None req-96652400-e204-4c5c-b6f4-074fc96f555f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 11 11:04:34.753020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96652400-e204-4c5c-b6f4-074fc96f555f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:34.757599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Jul 11 11:04:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:04:34.763572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:34.783877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a40c3f-453f-424e-881f-ec1d2d8b7120 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:34.789751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06d6e7-562e-44d8-8f17-ab39de5fb707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:34.790880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06d6e7-562e-44d8-8f17-ab39de5fb707 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:34.791168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06d6e7-562e-44d8-8f17-ab39de5fb707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:34.791472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06d6e7-562e-44d8-8f17-ab39de5fb707 admin admin] PATH_INFO: `/v3/projects/d16c61acfee74652b6cd4c256cd8f443` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:34.795629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06d6e7-562e-44d8-8f17-ab39de5fb707 admin admin] RBAC: Authorizing `identity:get_project(project_id=d16c61acfee74652b6cd4c256cd8f443)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:34.847867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING keystone.common.rbac_enforcer.enforcer [None req-da06d6e7-562e-44d8-8f17-ab39de5fb707 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 11 11:04:34.848206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06d6e7-562e-44d8-8f17-ab39de5fb707 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:34.852863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Mon Jul 11 11:04:34 2022] GET /identity/v3/projects/d16c61acfee74652b6cd4c256cd8f443 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:36.847515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa1ada-9aa0-4c2e-97ba-1ea8f6c11309 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:36.848049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa1ada-9aa0-4c2e-97ba-1ea8f6c11309 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:36.848134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa1ada-9aa0-4c2e-97ba-1ea8f6c11309 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:36.852158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:36.857460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310237-d88a-4c11-a8ec-5b6cb3e86c70 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:36.857894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310237-d88a-4c11-a8ec-5b6cb3e86c70 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:36.858085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310237-d88a-4c11-a8ec-5b6cb3e86c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:36.877657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d7310237-d88a-4c11-a8ec-5b6cb3e86c70 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:36.879180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d7310237-d88a-4c11-a8ec-5b6cb3e86c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:36.887042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 11 11:04:36.908936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa1ada-9aa0-4c2e-97ba-1ea8f6c11309 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:36.920681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa1ada-9aa0-4c2e-97ba-1ea8f6c11309 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:36.925320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147ae1c-918f-40ca-9ff9-e58ad16aadf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:36.925950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ae1c-918f-40ca-9ff9-e58ad16aadf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:36.926486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ae1c-918f-40ca-9ff9-e58ad16aadf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:36.926834 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147ae1c-918f-40ca-9ff9-e58ad16aadf8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:36.934438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147ae1c-918f-40ca-9ff9-e58ad16aadf8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:36.936230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147ae1c-918f-40ca-9ff9-e58ad16aadf8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:36.945604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Mon Jul 11 11:04:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:04:36.951456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310237-d88a-4c11-a8ec-5b6cb3e86c70 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:36.960512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310237-d88a-4c11-a8ec-5b6cb3e86c70 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:36.967293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f75b2f-da80-4aba-93bb-19d2141aa0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:36.968165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f75b2f-da80-4aba-93bb-19d2141aa0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:36.968448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f75b2f-da80-4aba-93bb-19d2141aa0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:36.968721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f75b2f-da80-4aba-93bb-19d2141aa0c2 admin admin] PATH_INFO: `/v3/users/41ca161a25324736922d4952cd76a1f3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:36.970142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f75b2f-da80-4aba-93bb-19d2141aa0c2 admin admin] RBAC: Authorizing `identity:get_user(user_id=41ca161a25324736922d4952cd76a1f3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:36.972024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f75b2f-da80-4aba-93bb-19d2141aa0c2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:36.973611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Mon Jul 11 11:04:36 2022] GET /identity/v3/users/41ca161a25324736922d4952cd76a1f3 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:39.091189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d0a1-81d2-4c7b-b6ad-c219b5fb5b9d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.091660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d0a1-81d2-4c7b-b6ad-c219b5fb5b9d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.091778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d0a1-81d2-4c7b-b6ad-c219b5fb5b9d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.093694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:04:39.098785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764e690-3014-427d-8495-1983d2db9ddb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.099168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764e690-3014-427d-8495-1983d2db9ddb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.099374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764e690-3014-427d-8495-1983d2db9ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.115549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3764e690-3014-427d-8495-1983d2db9ddb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:39.116707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3764e690-3014-427d-8495-1983d2db9ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:39.151702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316814e-180a-4ac1-a636-0c5260c7cae6 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.151994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316814e-180a-4ac1-a636-0c5260c7cae6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.152394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316814e-180a-4ac1-a636-0c5260c7cae6 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.153759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:04:39.160057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.160525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.160993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.197488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:39.200540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG passlib.handlers.bcrypt [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80500) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 11 11:04:39.200957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG passlib.handlers.bcrypt [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80500) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 11 11:04:39.221011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764e690-3014-427d-8495-1983d2db9ddb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:39.243520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:39.246907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:39.248528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:39.324589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764e690-3014-427d-8495-1983d2db9ddb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:39.346372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:04:39.357692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a455cf4-bdfd-4f76-8dc7-42ff93a0d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:39.358529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a455cf4-bdfd-4f76-8dc7-42ff93a0d35b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.358816 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a455cf4-bdfd-4f76-8dc7-42ff93a0d35b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.359096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a455cf4-bdfd-4f76-8dc7-42ff93a0d35b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.367861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:39.370254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a455cf4-bdfd-4f76-8dc7-42ff93a0d35b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4a455cf4-bdfd-4f76-8dc7-42ff93a0d35b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:39.381456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:39.382326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:04:39.383190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:04:39.388459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a455cf4-bdfd-4f76-8dc7-42ff93a0d35b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:39.467220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5598a-f0e7-41e2-a7b8-0f0fdf4d3c8e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:39.468600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:39.469583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:39.486219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a455cf4-bdfd-4f76-8dc7-42ff93a0d35b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:39.504139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125bea3-eb6a-4069-92e7-e25e106b734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:39.505043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125bea3-eb6a-4069-92e7-e25e106b734b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.505337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125bea3-eb6a-4069-92e7-e25e106b734b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.505623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125bea3-eb6a-4069-92e7-e25e106b734b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.519707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:39.520439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.520727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.521005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.521757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:39.522063 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:39.523867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:39.535954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:04:39.541515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:39.572692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125bea3-eb6a-4069-92e7-e25e106b734b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:39.574224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125bea3-eb6a-4069-92e7-e25e106b734b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:39.638302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0ccd0-dbfa-4cec-adb2-3a450fff7f29 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:39.669105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:04:39.673455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd982f75-bb0e-4963-a8b1-b324399b55ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:39.674500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982f75-bb0e-4963-a8b1-b324399b55ee None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.674694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125bea3-eb6a-4069-92e7-e25e106b734b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:39.674811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982f75-bb0e-4963-a8b1-b324399b55ee None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.675060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd982f75-bb0e-4963-a8b1-b324399b55ee None admin] PATH_INFO: `/v3/users/41ca161a25324736922d4952cd76a1f3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.676571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd982f75-bb0e-4963-a8b1-b324399b55ee None admin] RBAC: Authorizing `identity:get_user(user_id=41ca161a25324736922d4952cd76a1f3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:39.678249 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:39.678359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd982f75-bb0e-4963-a8b1-b324399b55ee None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:39.679972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:39 2022] GET /identity/v3/users/41ca161a25324736922d4952cd76a1f3 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:04:39.685132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd982f75-bb0e-4963-a8b1-b324399b55ee None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:39.699831 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:39.771943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125bea3-eb6a-4069-92e7-e25e106b734b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:39.782793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd982f75-bb0e-4963-a8b1-b324399b55ee None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:39.805373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:39.806158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.806439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.806715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.818222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebc1d2-d0f7-4a90-a5f6-246e337cdd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:39.819012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebc1d2-d0f7-4a90-a5f6-246e337cdd5c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.819326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebc1d2-d0f7-4a90-a5f6-246e337cdd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.819609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebc1d2-d0f7-4a90-a5f6-246e337cdd5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.820799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebc1d2-d0f7-4a90-a5f6-246e337cdd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:39.822249 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebc1d2-d0f7-4a90-a5f6-246e337cdd5c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:39.824075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:04:39.825113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:39.826559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:39.830190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebc1d2-d0f7-4a90-a5f6-246e337cdd5c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:04:39.851121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:39.852038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:39.852840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:04:39.852840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:04:39.852840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:04:39.852840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:04:39.852840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:04:39.858202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:39.909010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:39.915499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:39.927908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebc1d2-d0f7-4a90-a5f6-246e337cdd5c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:39.958175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5749373-d09d-4302-a152-5eb90c1a5aaf None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:39.961741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83814a15-87de-4a0d-a1a6-72b4cddda07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:39.962401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83814a15-87de-4a0d-a1a6-72b4cddda07f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.962698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83814a15-87de-4a0d-a1a6-72b4cddda07f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.962979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83814a15-87de-4a0d-a1a6-72b4cddda07f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.963852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83814a15-87de-4a0d-a1a6-72b4cddda07f None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.domain.id=default, user.id=41ca161a25324736922d4952cd76a1f3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:39.964186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83814a15-87de-4a0d-a1a6-72b4cddda07f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:39.965614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83814a15-87de-4a0d-a1a6-72b4cddda07f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:39.972757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Jul 11 11:04:39 2022] GET /identity/v3/role_assignments?user.id=41ca161a25324736922d4952cd76a1f3&scope.domain.id=default&role.id=007af6bd1b734517b934b4cac45faf2c => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:04:39.992816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a44af9-a8d6-4cd9-919d-506ee7ac5c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:39.993505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a44af9-a8d6-4cd9-919d-506ee7ac5c51 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:39.993792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a44af9-a8d6-4cd9-919d-506ee7ac5c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:39.994073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a44af9-a8d6-4cd9-919d-506ee7ac5c51 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:39.994828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a44af9-a8d6-4cd9-919d-506ee7ac5c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:39.995141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a44af9-a8d6-4cd9-919d-506ee7ac5c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:39.996613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a44af9-a8d6-4cd9-919d-506ee7ac5c51 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:40.016274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:04:42.152933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7d1ae-da41-4eea-8aae-eddbf72fe2b6 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.153924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7d1ae-da41-4eea-8aae-eddbf72fe2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.154401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7d1ae-da41-4eea-8aae-eddbf72fe2b6 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.156056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:42.161361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8a1f8-5ee6-465e-9070-6b12b01acf66 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.162160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8a1f8-5ee6-465e-9070-6b12b01acf66 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.162632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8a1f8-5ee6-465e-9070-6b12b01acf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.181123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bfc8a1f8-5ee6-465e-9070-6b12b01acf66 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:42.182613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bfc8a1f8-5ee6-465e-9070-6b12b01acf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:42.258045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:42.280805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7d1ae-da41-4eea-8aae-eddbf72fe2b6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:42.424915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7d1ae-da41-4eea-8aae-eddbf72fe2b6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:42.471508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b01d01-65d4-4dc3-8842-966a4f4f3bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:42.472308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01d01-65d4-4dc3-8842-966a4f4f3bef None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.472594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01d01-65d4-4dc3-8842-966a4f4f3bef None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.472870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b01d01-65d4-4dc3-8842-966a4f4f3bef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.492135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b01d01-65d4-4dc3-8842-966a4f4f3bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-94b01d01-65d4-4dc3-8842-966a4f4f3bef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:42.500384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:42.501219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:04:42.502001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 13/24] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:04:42.510690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8a1f8-5ee6-465e-9070-6b12b01acf66 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:42.570139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:42.625356 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0dfe-4257-4cd4-85ab-644c82b88fdb None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.625669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0dfe-4257-4cd4-85ab-644c82b88fdb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.625928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e0dfe-4257-4cd4-85ab-644c82b88fdb None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.631081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8a1f8-5ee6-465e-9070-6b12b01acf66 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:42.632052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 14/25] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:42.637218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5433-cdd2-4417-904b-66007b285bed None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.637506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5433-cdd2-4417-904b-66007b285bed None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.637793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a5433-cdd2-4417-904b-66007b285bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.651306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:42.669252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:42.671384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.671384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.671384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.672015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:42.672468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-315a5433-cdd2-4417-904b-66007b285bed None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:42.672675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:42.673807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-315a5433-cdd2-4417-904b-66007b285bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:42.674534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:42.698857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 12/26] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:42.708554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:42.793690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 15/27] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:04:42.800751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f4f88-6832-4a42-bb09-d46f20cf563a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.801303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f4f88-6832-4a42-bb09-d46f20cf563a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.801589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f4f88-6832-4a42-bb09-d46f20cf563a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.802943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 16/28] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:04:42.815958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec82cf-cf16-4a4e-accd-bd362a51a4ad None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.816252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec82cf-cf16-4a4e-accd-bd362a51a4ad None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.816540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec82cf-cf16-4a4e-accd-bd362a51a4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.849056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-87ec82cf-cf16-4a4e-accd-bd362a51a4ad None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:42.850271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-87ec82cf-cf16-4a4e-accd-bd362a51a4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:42.917840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb707dc8-e5bc-48ca-aa41-bc0ae91a889e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:42.931700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:42.975978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b161-f923-4e6a-a452-ea174116bea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:42.978347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b161-f923-4e6a-a452-ea174116bea7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:42.979561 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b161-f923-4e6a-a452-ea174116bea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:42.980614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8b161-f923-4e6a-a452-ea174116bea7 None admin] PATH_INFO: `/v3/users/41ca161a25324736922d4952cd76a1f3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:42.982575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8b161-f923-4e6a-a452-ea174116bea7 None admin] RBAC: Authorizing `identity:get_user(user_id=41ca161a25324736922d4952cd76a1f3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:42.987086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8b161-f923-4e6a-a452-ea174116bea7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:42.988599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 17/29] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:04:42.990468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 13/30] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:42 2022] GET /identity/v3/users/41ca161a25324736922d4952cd76a1f3 => generated 247 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:04:42.991588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec82cf-cf16-4a4e-accd-bd362a51a4ad None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:43.000484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b161-f923-4e6a-a452-ea174116bea7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:43.115567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8b161-f923-4e6a-a452-ea174116bea7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:43.132455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:43.153739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564f08-4b69-48a0-8065-31a371c82505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:43.154797 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564f08-4b69-48a0-8065-31a371c82505 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.155295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564f08-4b69-48a0-8065-31a371c82505 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.155711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564f08-4b69-48a0-8065-31a371c82505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.157075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564f08-4b69-48a0-8065-31a371c82505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:43.158677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564f08-4b69-48a0-8065-31a371c82505 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:43.160483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 14/31] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:43.161357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564f08-4b69-48a0-8065-31a371c82505 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:43.211395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec82cf-cf16-4a4e-accd-bd362a51a4ad None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:43.212751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:43.260007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:43.261047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.261493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.261916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.265610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564f08-4b69-48a0-8065-31a371c82505 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:43.277746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:43.279156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:43.302067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f89dd-7e19-4f04-90a5-ec705e756b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:43.303110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f89dd-7e19-4f04-90a5-ec705e756b34 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.303656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f89dd-7e19-4f04-90a5-ec705e756b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.304121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f89dd-7e19-4f04-90a5-ec705e756b34 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.305121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f89dd-7e19-4f04-90a5-ec705e756b34 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:43.306753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f89dd-7e19-4f04-90a5-ec705e756b34 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:04:43.314436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:43.315357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:43.316160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:04:43.316160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:04:43.316160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:04:43.316160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:04:43.320160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 18/32] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:04:43.321261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:43.458891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75f292-3920-4da1-84f6-59a4dcfc03e8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:43.478350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:43.501410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4aa6d9-f89f-4964-ada4-c2bc7d3e63fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:43.504464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4aa6d9-f89f-4964-ada4-c2bc7d3e63fd None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.504772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4aa6d9-f89f-4964-ada4-c2bc7d3e63fd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.505068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4aa6d9-f89f-4964-ada4-c2bc7d3e63fd None admin] PATH_INFO: `/v3/domains/default/users/41ca161a25324736922d4952cd76a1f3/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.507437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4aa6d9-f89f-4964-ada4-c2bc7d3e63fd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=41ca161a25324736922d4952cd76a1f3, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:43.509423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4aa6d9-f89f-4964-ada4-c2bc7d3e63fd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:43.563069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 15/33] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 11 11:04:43 2022] POST /identity/v3/roles => generated 219 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:43.564061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f89dd-7e19-4f04-90a5-ec705e756b34 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:43.682841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f89dd-7e19-4f04-90a5-ec705e756b34 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:43.686565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:43.723041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c03038-e3aa-4fd3-85b1-8104f85ab6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:43.724260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03038-e3aa-4fd3-85b1-8104f85ab6fd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.724871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03038-e3aa-4fd3-85b1-8104f85ab6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.725814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03038-e3aa-4fd3-85b1-8104f85ab6fd None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.726766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c03038-e3aa-4fd3-85b1-8104f85ab6fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:43.727081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c03038-e3aa-4fd3-85b1-8104f85ab6fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:43.728642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c03038-e3aa-4fd3-85b1-8104f85ab6fd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:43.744133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 19/34] 10.222.0.26 () {58 vars in 1361 bytes} [Mon Jul 11 11:04:43 2022] PUT /identity/v3/domains/default/users/41ca161a25324736922d4952cd76a1f3/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:04:43.745077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 16/35] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:43.746495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb31c4-29e2-435f-8d59-6bdcedf868c1 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.746623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3785b0-d716-4b57-8827-6c79ccf01a0e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.746831 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb31c4-29e2-435f-8d59-6bdcedf868c1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.746925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3785b0-d716-4b57-8827-6c79ccf01a0e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.747073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb31c4-29e2-435f-8d59-6bdcedf868c1 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.747159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3785b0-d716-4b57-8827-6c79ccf01a0e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.748797 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 17/36] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:43.754411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6687ba-c752-49bd-9a20-47ad23acb44b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.755640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 20/37] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:43.756741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb31c4-29e2-435f-8d59-6bdcedf868c1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:43.757828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6687ba-c752-49bd-9a20-47ad23acb44b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.758760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6687ba-c752-49bd-9a20-47ad23acb44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.795007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3b6687ba-c752-49bd-9a20-47ad23acb44b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:43.797500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3b6687ba-c752-49bd-9a20-47ad23acb44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:43.881236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 18/38] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:43.883726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76929e1f-9f7c-48df-bead-a090c2826bd0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.883726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76929e1f-9f7c-48df-bead-a090c2826bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.884673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76929e1f-9f7c-48df-bead-a090c2826bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.901326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb31c4-29e2-435f-8d59-6bdcedf868c1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:43.921392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-76929e1f-9f7c-48df-bead-a090c2826bd0 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:43.922722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-76929e1f-9f7c-48df-bead-a090c2826bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:43.950844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec879f-6b3d-4916-bc65-8a7531a7ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:43.951800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec879f-6b3d-4916-bc65-8a7531a7ea37 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:43.952248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec879f-6b3d-4916-bc65-8a7531a7ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:43.952663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec879f-6b3d-4916-bc65-8a7531a7ea37 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:43.954155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec879f-6b3d-4916-bc65-8a7531a7ea37 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:43.954987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec879f-6b3d-4916-bc65-8a7531a7ea37 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:44.003463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 19/39] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:44.004305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76929e1f-9f7c-48df-bead-a090c2826bd0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:44.029128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:44.089422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:04:43 2022] POST /identity/v3/projects => generated 301 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:44.090809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831da95-af18-4e59-afc7-c924f1dc01ae None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:44.091109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831da95-af18-4e59-afc7-c924f1dc01ae None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:44.091422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831da95-af18-4e59-afc7-c924f1dc01ae None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:44.093418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 22/41] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:44.096389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831da95-af18-4e59-afc7-c924f1dc01ae None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:44.121072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76929e1f-9f7c-48df-bead-a090c2826bd0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:44.137863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:44.172456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ed898-3583-49fd-b9cb-9a9d5fdd6151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:44.173311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ed898-3583-49fd-b9cb-9a9d5fdd6151 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:44.173609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ed898-3583-49fd-b9cb-9a9d5fdd6151 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:44.173892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ed898-3583-49fd-b9cb-9a9d5fdd6151 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:44.175109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ed898-3583-49fd-b9cb-9a9d5fdd6151 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:44.177183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ed898-3583-49fd-b9cb-9a9d5fdd6151 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:44.217164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831da95-af18-4e59-afc7-c924f1dc01ae None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:44.260190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 20/42] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 11 11:04:44 2022] POST /identity/v3/roles => generated 223 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:44.267274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4693-a497-4ecd-84ec-f1dcf643c019 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:44.267274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4693-a497-4ecd-84ec-f1dcf643c019 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:44.267274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4693-a497-4ecd-84ec-f1dcf643c019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:44.267635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0c609-d342-4812-8ade-77e5e097d4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:44.267635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0c609-d342-4812-8ade-77e5e097d4b0 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:44.267635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0c609-d342-4812-8ade-77e5e097d4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:44.267635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0c609-d342-4812-8ade-77e5e097d4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:44.268027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0c609-d342-4812-8ade-77e5e097d4b0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:44.268664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0c609-d342-4812-8ade-77e5e097d4b0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:44.290735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fc4b4693-a497-4ecd-84ec-f1dcf643c019 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:44.292085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fc4b4693-a497-4ecd-84ec-f1dcf643c019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:44.356695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 23/43] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 11 11:04:44 2022] POST /identity/v3/roles => generated 225 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:44.384645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 21/44] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:44.424482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0c609-d342-4812-8ade-77e5e097d4b0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:44.636183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:44.648714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0c609-d342-4812-8ade-77e5e097d4b0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:44.682181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367a64a-2f91-4383-acce-22d953ea9efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:44.682906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a64a-2f91-4383-acce-22d953ea9efc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:44.683190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a64a-2f91-4383-acce-22d953ea9efc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:44.683731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a64a-2f91-4383-acce-22d953ea9efc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:44.684906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367a64a-2f91-4383-acce-22d953ea9efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:44.686882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367a64a-2f91-4383-acce-22d953ea9efc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:44.688849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 24/45] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:44.693929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b4693-a497-4ecd-84ec-f1dcf643c019 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:44.709552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:44.791052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b4693-a497-4ecd-84ec-f1dcf643c019 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:44.833295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d79a31-bd67-4714-9d4f-cb82288d3100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:44.833970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d79a31-bd67-4714-9d4f-cb82288d3100 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:44.834257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d79a31-bd67-4714-9d4f-cb82288d3100 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:44.834539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d79a31-bd67-4714-9d4f-cb82288d3100 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:44.835275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d79a31-bd67-4714-9d4f-cb82288d3100 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:44.836846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d79a31-bd67-4714-9d4f-cb82288d3100 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:44.938366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 22/46] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:04:44 2022] POST /identity/v3/projects => generated 312 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:45.837581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff245cfb-2584-494f-ba58-0e8626dc7566 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:45.838113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff245cfb-2584-494f-ba58-0e8626dc7566 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:45.838196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff245cfb-2584-494f-ba58-0e8626dc7566 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:45.839548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 25/47] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:45.844770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc5e27-5a23-4b80-a2ef-3544f35bee0e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:45.845196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc5e27-5a23-4b80-a2ef-3544f35bee0e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:45.845334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc5e27-5a23-4b80-a2ef-3544f35bee0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:45.862101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dfbc5e27-5a23-4b80-a2ef-3544f35bee0e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:45.863305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dfbc5e27-5a23-4b80-a2ef-3544f35bee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:45.920207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:45.928129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 23/48] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:45.950904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff245cfb-2584-494f-ba58-0e8626dc7566 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:46.057332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff245cfb-2584-494f-ba58-0e8626dc7566 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:46.090957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5ecf-b000-4812-9ced-721ba3ee1863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:46.091744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5ecf-b000-4812-9ced-721ba3ee1863 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:46.092031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5ecf-b000-4812-9ced-721ba3ee1863 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:46.092310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f5ecf-b000-4812-9ced-721ba3ee1863 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:46.102621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f5ecf-b000-4812-9ced-721ba3ee1863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-cc7f5ecf-b000-4812-9ced-721ba3ee1863 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:46.110811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:46.111877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:04:46.112958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 26/49] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:04:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:04:46.117899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc5e27-5a23-4b80-a2ef-3544f35bee0e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:46.212981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc5e27-5a23-4b80-a2ef-3544f35bee0e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:46.349917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:46.350496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:46.350781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:46.351059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:46.351847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:46.352163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:46.353525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:46.361980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 24/50] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:46.367441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5ecf-b000-4812-9ced-721ba3ee1863 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:46.385531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:46.471033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f5ecf-b000-4812-9ced-721ba3ee1863 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:46.504644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523f6d7-c916-490a-ad50-52569eee5286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:46.505518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523f6d7-c916-490a-ad50-52569eee5286 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:46.505811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523f6d7-c916-490a-ad50-52569eee5286 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:46.506091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523f6d7-c916-490a-ad50-52569eee5286 None admin] PATH_INFO: `/v3/users/41ca161a25324736922d4952cd76a1f3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:46.507325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523f6d7-c916-490a-ad50-52569eee5286 None admin] RBAC: Authorizing `identity:get_user(user_id=41ca161a25324736922d4952cd76a1f3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:46.508795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523f6d7-c916-490a-ad50-52569eee5286 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:46.510448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 27/51] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:46 2022] GET /identity/v3/users/41ca161a25324736922d4952cd76a1f3 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:46.515996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:46.572653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:46.613883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1075ac4-ae26-45a1-a226-0d6f2df9cd30 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:46.646477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31706214-a5d2-49bd-a854-bb46ec228c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:46.647292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31706214-a5d2-49bd-a854-bb46ec228c67 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:46.647585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31706214-a5d2-49bd-a854-bb46ec228c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:46.647866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31706214-a5d2-49bd-a854-bb46ec228c67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:46.649041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31706214-a5d2-49bd-a854-bb46ec228c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:46.650501 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31706214-a5d2-49bd-a854-bb46ec228c67 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:46.652167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 25/52] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:46.658391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523f6d7-c916-490a-ad50-52569eee5286 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:46.741191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:46.755944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523f6d7-c916-490a-ad50-52569eee5286 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:46.798049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d41984-aad0-479f-8fe4-2fe46d73461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:46.798713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d41984-aad0-479f-8fe4-2fe46d73461f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:46.799026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d41984-aad0-479f-8fe4-2fe46d73461f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:46.799343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d41984-aad0-479f-8fe4-2fe46d73461f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:46.800189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d41984-aad0-479f-8fe4-2fe46d73461f None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.domain.id=default, user.id=41ca161a25324736922d4952cd76a1f3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:46.800504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d41984-aad0-479f-8fe4-2fe46d73461f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:46.802228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d41984-aad0-479f-8fe4-2fe46d73461f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:46.808507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 28/53] 10.222.0.26 () {56 vars in 1322 bytes} [Mon Jul 11 11:04:46 2022] GET /identity/v3/role_assignments?user.id=41ca161a25324736922d4952cd76a1f3&scope.domain.id=default&role.id=007af6bd1b734517b934b4cac45faf2c => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:04:46.948488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f3724-0084-4fcc-9c59-703613634347 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:46.949010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f3724-0084-4fcc-9c59-703613634347 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:46.949119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f3724-0084-4fcc-9c59-703613634347 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:46.950409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 26/54] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:46.956254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d353c-6fba-4131-ae29-0bbf370b5d4b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:46.956650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d353c-6fba-4131-ae29-0bbf370b5d4b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:46.956822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d353c-6fba-4131-ae29-0bbf370b5d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:46.973769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-711d353c-6fba-4131-ae29-0bbf370b5d4b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:46.975319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-711d353c-6fba-4131-ae29-0bbf370b5d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:47.040713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 29/55] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:47.063934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f3724-0084-4fcc-9c59-703613634347 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:47.170447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f3724-0084-4fcc-9c59-703613634347 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:47.194849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:47.203568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d5cbaf-2a13-407e-8013-e76f7b6f7bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:47.204305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5cbaf-2a13-407e-8013-e76f7b6f7bce None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:47.204594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5cbaf-2a13-407e-8013-e76f7b6f7bce None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:47.204873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5cbaf-2a13-407e-8013-e76f7b6f7bce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:47.206047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5cbaf-2a13-407e-8013-e76f7b6f7bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:47.207519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d5cbaf-2a13-407e-8013-e76f7b6f7bce None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:47.209132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 27/56] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:47.214733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d353c-6fba-4131-ae29-0bbf370b5d4b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:47.290611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:47.311575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d353c-6fba-4131-ae29-0bbf370b5d4b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:47.345138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f47d4-cbf9-4ccf-a23e-f45b8671d65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:47.345785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f47d4-cbf9-4ccf-a23e-f45b8671d65f None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:47.346071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f47d4-cbf9-4ccf-a23e-f45b8671d65f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:47.346349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f47d4-cbf9-4ccf-a23e-f45b8671d65f None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:47.347053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f47d4-cbf9-4ccf-a23e-f45b8671d65f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:47.348542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f47d4-cbf9-4ccf-a23e-f45b8671d65f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:47.429470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 30/57] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Jul 11 11:04:47 2022] POST /identity/v3/projects => generated 298 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:49.485481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39e77-f31d-4b0f-bb94-4d1dddc042a6 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:49.486038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39e77-f31d-4b0f-bb94-4d1dddc042a6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:49.486038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b39e77-f31d-4b0f-bb94-4d1dddc042a6 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:49.487441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 28/58] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:49.492660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b0295-1925-4e37-9751-e7b79f94b35f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:49.493077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b0295-1925-4e37-9751-e7b79f94b35f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:49.493224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b0295-1925-4e37-9751-e7b79f94b35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:49.509428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5e1b0295-1925-4e37-9751-e7b79f94b35f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:49.510613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5e1b0295-1925-4e37-9751-e7b79f94b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:49.574294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 31/59] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:49.598965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b39e77-f31d-4b0f-bb94-4d1dddc042a6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:49.696520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b39e77-f31d-4b0f-bb94-4d1dddc042a6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:49.728790 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4128f0-d6bd-442d-b3fe-1a35098f731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:49.729570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4128f0-d6bd-442d-b3fe-1a35098f731b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:49.729854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4128f0-d6bd-442d-b3fe-1a35098f731b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:49.730130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4128f0-d6bd-442d-b3fe-1a35098f731b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:49.731316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4128f0-d6bd-442d-b3fe-1a35098f731b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:49.732747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4128f0-d6bd-442d-b3fe-1a35098f731b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:49.734333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 29/60] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:49.739784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b0295-1925-4e37-9751-e7b79f94b35f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:49.797926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:49.837375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b0295-1925-4e37-9751-e7b79f94b35f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:49.870418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f6cee-3e51-488f-9c24-c75a842d0013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:49.871076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6cee-3e51-488f-9c24-c75a842d0013 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:49.871390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6cee-3e51-488f-9c24-c75a842d0013 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:49.871671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f6cee-3e51-488f-9c24-c75a842d0013 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:49.872391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f6cee-3e51-488f-9c24-c75a842d0013 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:49.873854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f6cee-3e51-488f-9c24-c75a842d0013 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:49.982142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 32/61] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:04:49 2022] POST /identity/v3/users => generated 275 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:53.075937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b93907-72a7-473b-bc41-127c490aa541 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.076870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b93907-72a7-473b-bc41-127c490aa541 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.077348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b93907-72a7-473b-bc41-127c490aa541 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.078939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 30/62] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:53.084231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32204bc6-b423-4ab6-87dd-a391fa253005 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.084650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32204bc6-b423-4ab6-87dd-a391fa253005 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.084814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32204bc6-b423-4ab6-87dd-a391fa253005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.105357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-32204bc6-b423-4ab6-87dd-a391fa253005 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:53.106550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-32204bc6-b423-4ab6-87dd-a391fa253005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:53.159751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34baf-7b5c-46b1-9617-e9b1d3df91b4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.160384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34baf-7b5c-46b1-9617-e9b1d3df91b4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.160853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34baf-7b5c-46b1-9617-e9b1d3df91b4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.162406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 31/63] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:53.175017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75466668-4f29-4c6f-b148-f598e888e0a7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.175607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75466668-4f29-4c6f-b148-f598e888e0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.176349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75466668-4f29-4c6f-b148-f598e888e0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.200942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-75466668-4f29-4c6f-b148-f598e888e0a7 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:53.204171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-75466668-4f29-4c6f-b148-f598e888e0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:53.218410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 33/64] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:53.254804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32204bc6-b423-4ab6-87dd-a391fa253005 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:53.286944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:53.298355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 32/65] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:04:53.321112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:53.328458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75466668-4f29-4c6f-b148-f598e888e0a7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:53.387959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32204bc6-b423-4ab6-87dd-a391fa253005 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:53.435483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a6e40-77ba-4ff9-8385-7aaea0d9787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:53.437065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a6e40-77ba-4ff9-8385-7aaea0d9787a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.437906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a6e40-77ba-4ff9-8385-7aaea0d9787a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.438341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a6e40-77ba-4ff9-8385-7aaea0d9787a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.453128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a6e40-77ba-4ff9-8385-7aaea0d9787a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:53.460763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75466668-4f29-4c6f-b148-f598e888e0a7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-cd7a6e40-77ba-4ff9-8385-7aaea0d9787a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:53.463094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:53.464164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:04:53.465241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 34/66] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:04:53.466018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4896-c193-476e-8f4b-be9361027639 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.466018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4896-c193-476e-8f4b-be9361027639 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.466173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4896-c193-476e-8f4b-be9361027639 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.472453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 35/67] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:53.472453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b4896-c193-476e-8f4b-be9361027639 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:53.516852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da3737-94ac-4be7-85ac-705661136f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:53.517653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da3737-94ac-4be7-85ac-705661136f61 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.517938 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da3737-94ac-4be7-85ac-705661136f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.518213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da3737-94ac-4be7-85ac-705661136f61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.533755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da3737-94ac-4be7-85ac-705661136f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a2da3737-94ac-4be7-85ac-705661136f61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:53.546756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:53.547728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:04:53.548504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 33/68] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:04:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:04:53.549229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f5c34-60e4-413e-a67e-02d41415a742 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.554364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f5c34-60e4-413e-a67e-02d41415a742 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.554364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f5c34-60e4-413e-a67e-02d41415a742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.571108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-179f5c34-60e4-413e-a67e-02d41415a742 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:53.572365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-179f5c34-60e4-413e-a67e-02d41415a742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:53.600401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b4896-c193-476e-8f4b-be9361027639 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:53.611354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:53.624344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:53.647927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:53.648894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.649350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.650922 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.651442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 34/69] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:53.652026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:53.652445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:53.652907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f5c34-60e4-413e-a67e-02d41415a742 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:53.654042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:53.664812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 36/70] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:04:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:04:53.671447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:53.766809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f5c34-60e4-413e-a67e-02d41415a742 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:53.786110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054b18a-9ab9-4e79-8115-fda6a99ee84c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:53.798909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:53.799611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.799897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.800189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.801192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:53.801502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:53.802883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:53.810687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:53.812026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:53.823350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349777e-cb83-4d93-9276-66f10b80af82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:53.824232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349777e-cb83-4d93-9276-66f10b80af82 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.824521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349777e-cb83-4d93-9276-66f10b80af82 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.824799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349777e-cb83-4d93-9276-66f10b80af82 None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.837936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349777e-cb83-4d93-9276-66f10b80af82 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:53.839419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349777e-cb83-4d93-9276-66f10b80af82 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:53.841081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 37/72] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:53 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:53.844571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090c88-336d-41b1-92bb-0c4e582d61bc None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.844862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090c88-336d-41b1-92bb-0c4e582d61bc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.845151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a090c88-336d-41b1-92bb-0c4e582d61bc None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.846417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 38/73] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:53.847566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033771d5-547a-470d-92bc-31b587daa6ec None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.848386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033771d5-547a-470d-92bc-31b587daa6ec None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.848822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033771d5-547a-470d-92bc-31b587daa6ec None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.850357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 39/74] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:53.851286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033771d5-547a-470d-92bc-31b587daa6ec None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:53.862551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:53.880588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:53.915007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429f325-02ac-4930-925f-2ba6f74e3ab7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:53.948156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440cd0-7033-47b5-a2ee-fabc25a2bb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:53.948968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440cd0-7033-47b5-a2ee-fabc25a2bb72 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.949255 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440cd0-7033-47b5-a2ee-fabc25a2bb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.949534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440cd0-7033-47b5-a2ee-fabc25a2bb72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.953601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033771d5-547a-470d-92bc-31b587daa6ec None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:53.960034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5440cd0-7033-47b5-a2ee-fabc25a2bb72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f5440cd0-7033-47b5-a2ee-fabc25a2bb72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:53.968099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:53.969011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:04:53.969875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 36/75] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:04:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:04:53.970839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440cd0-7033-47b5-a2ee-fabc25a2bb72 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:53.990609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9952c8-c1b7-44b1-a1c4-0ffb066a588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:53.991592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9952c8-c1b7-44b1-a1c4-0ffb066a588a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.995539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9952c8-c1b7-44b1-a1c4-0ffb066a588a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.995539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9952c8-c1b7-44b1-a1c4-0ffb066a588a None admin] PATH_INFO: `/v3/users/41ca161a25324736922d4952cd76a1f3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:53.995539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9952c8-c1b7-44b1-a1c4-0ffb066a588a None admin] RBAC: Authorizing `identity:get_user(user_id=41ca161a25324736922d4952cd76a1f3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:53.995539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9952c8-c1b7-44b1-a1c4-0ffb066a588a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:53.996413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 40/76] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:53 2022] GET /identity/v3/users/41ca161a25324736922d4952cd76a1f3 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:53.997546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028953ae-4511-4a04-ac43-128141fcc2a7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:53.997845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028953ae-4511-4a04-ac43-128141fcc2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:53.998109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028953ae-4511-4a04-ac43-128141fcc2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.014265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-028953ae-4511-4a04-ac43-128141fcc2a7 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:54.015501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-028953ae-4511-4a04-ac43-128141fcc2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:54.067602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440cd0-7033-47b5-a2ee-fabc25a2bb72 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.070153 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.082275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 41/77] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:54.083606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc616-59fe-4ad8-972e-f2668153737f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.083899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc616-59fe-4ad8-972e-f2668153737f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.084292 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc616-59fe-4ad8-972e-f2668153737f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.101384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ffdfc616-59fe-4ad8-972e-f2668153737f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:54.102699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ffdfc616-59fe-4ad8-972e-f2668153737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:54.102970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff1ba-26db-448d-890f-13a032b8ea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.103652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff1ba-26db-448d-890f-13a032b8ea4e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.103941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff1ba-26db-448d-890f-13a032b8ea4e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.104218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff1ba-26db-448d-890f-13a032b8ea4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.120522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.169367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 42/78] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:54.170231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc616-59fe-4ad8-972e-f2668153737f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.177497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ff1ba-26db-448d-890f-13a032b8ea4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:54.178991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ff1ba-26db-448d-890f-13a032b8ea4e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:54.267635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc616-59fe-4ad8-972e-f2668153737f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.274160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 37/79] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:04:54.275100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff1ba-26db-448d-890f-13a032b8ea4e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.278109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.304775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.305449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.305734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.306010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.306765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:54.307078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:54.308507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:54.316309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 43/80] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:54.317090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.338623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.372219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff1ba-26db-448d-890f-13a032b8ea4e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.407295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7d6f5-1c23-4b81-99c9-e4b5274763ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.407945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7d6f5-1c23-4b81-99c9-e4b5274763ed None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.408237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7d6f5-1c23-4b81-99c9-e4b5274763ed None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.408519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7d6f5-1c23-4b81-99c9-e4b5274763ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.418520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa850d8-6881-4e6e-b519-57eab5fdb4e8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.451322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f786fa-4e9d-4562-be57-a14345d91186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.452678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f786fa-4e9d-4562-be57-a14345d91186 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.452963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f786fa-4e9d-4562-be57-a14345d91186 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.453244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f786fa-4e9d-4562-be57-a14345d91186 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.463027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f786fa-4e9d-4562-be57-a14345d91186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f4f786fa-4e9d-4562-be57-a14345d91186 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:54.470677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:54.471731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 44/81] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:04:54.477582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f786fa-4e9d-4562-be57-a14345d91186 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.478282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7d6f5-1c23-4b81-99c9-e4b5274763ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:54.478282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7d6f5-1c23-4b81-99c9-e4b5274763ed None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:54.479812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.556757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.573537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 38/82] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:04:54.574243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7d6f5-1c23-4b81-99c9-e4b5274763ed None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.580186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f786fa-4e9d-4562-be57-a14345d91186 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.612975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c30ba-5c3b-45b5-9446-a54285e72e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.613787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c30ba-5c3b-45b5-9446-a54285e72e91 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.614070 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c30ba-5c3b-45b5-9446-a54285e72e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.614346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c30ba-5c3b-45b5-9446-a54285e72e91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.615556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c30ba-5c3b-45b5-9446-a54285e72e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:54.617003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c30ba-5c3b-45b5-9446-a54285e72e91 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:54.618633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 45/83] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:54.619420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c30ba-5c3b-45b5-9446-a54285e72e91 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.669895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7d6f5-1c23-4b81-99c9-e4b5274763ed None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.673172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.703715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c41c7f-815c-4097-953d-95a4f25ab060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.704580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c41c7f-815c-4097-953d-95a4f25ab060 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.704861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c41c7f-815c-4097-953d-95a4f25ab060 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.705136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c41c7f-815c-4097-953d-95a4f25ab060 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.706321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c41c7f-815c-4097-953d-95a4f25ab060 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:54.707854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c41c7f-815c-4097-953d-95a4f25ab060 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:54.709975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 39/84] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:54.710869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c41c7f-815c-4097-953d-95a4f25ab060 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.721815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c30ba-5c3b-45b5-9446-a54285e72e91 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.754774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a1e7e-9363-40bb-adf9-fa70b58eabf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.755681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a1e7e-9363-40bb-adf9-fa70b58eabf7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.755967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a1e7e-9363-40bb-adf9-fa70b58eabf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.756248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a1e7e-9363-40bb-adf9-fa70b58eabf7 None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.757449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a1e7e-9363-40bb-adf9-fa70b58eabf7 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:54.758890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a1e7e-9363-40bb-adf9-fa70b58eabf7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:54.760629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 46/85] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:54.761384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a1e7e-9363-40bb-adf9-fa70b58eabf7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.775705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.812076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c41c7f-815c-4097-953d-95a4f25ab060 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.845284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.845950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.846239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.846520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.847276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:54.847600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:54.848961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:54.856808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 40/86] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:54.857576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.863835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a1e7e-9363-40bb-adf9-fa70b58eabf7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.896416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:54.900653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10345976-6c0c-416d-ac4e-59f2b510fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:54.901520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10345976-6c0c-416d-ac4e-59f2b510fd36 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:54.901809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10345976-6c0c-416d-ac4e-59f2b510fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:54.902090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10345976-6c0c-416d-ac4e-59f2b510fd36 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:54.903317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10345976-6c0c-416d-ac4e-59f2b510fd36 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:54.905336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10345976-6c0c-416d-ac4e-59f2b510fd36 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:54.907049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 47/87] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:54.907847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10345976-6c0c-416d-ac4e-59f2b510fd36 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:54.974127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036371-96cf-4c98-bbe4-4581f07c3295 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:54.987145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:55.005413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10345976-6c0c-416d-ac4e-59f2b510fd36 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:55.006225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b9197-3348-4d1a-bc0a-0db78e2c083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:55.006893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b9197-3348-4d1a-bc0a-0db78e2c083c None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:55.007182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b9197-3348-4d1a-bc0a-0db78e2c083c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:55.007491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b9197-3348-4d1a-bc0a-0db78e2c083c None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:55.008208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b9197-3348-4d1a-bc0a-0db78e2c083c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:55.009734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b9197-3348-4d1a-bc0a-0db78e2c083c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:55.039242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:55.039918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:55.040203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:55.040480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:55.041303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] RBAC: Adding query filter params (role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=6ca74ec2a86e48a28a003d344862f755, user.id=0d3b8b0ba0b84fecbef6319c3179a020) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:55.041615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:55.043046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:55.048493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 48/88] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:04:54 2022] GET /identity/v3/role_assignments?user.id=0d3b8b0ba0b84fecbef6319c3179a020&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:55.049261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:55.111633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 41/89] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:04:54 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:55.111633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b9197-3348-4d1a-bc0a-0db78e2c083c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:55.191893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada61541-886c-417c-bf7f-bd2c8f43780c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:55.222193 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:55.237824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29c76b-fb59-41d8-93ce-ad7f5d281b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:55.238403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29c76b-fb59-41d8-93ce-ad7f5d281b27 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:55.238688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29c76b-fb59-41d8-93ce-ad7f5d281b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:55.238965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29c76b-fb59-41d8-93ce-ad7f5d281b27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:55.240179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b9197-3348-4d1a-bc0a-0db78e2c083c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:55.258965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:55.298086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825171b-89f6-4580-bdbb-3f2aa0532939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:55.298978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825171b-89f6-4580-bdbb-3f2aa0532939 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:55.299282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825171b-89f6-4580-bdbb-3f2aa0532939 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:55.299614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825171b-89f6-4580-bdbb-3f2aa0532939 None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:55.301240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825171b-89f6-4580-bdbb-3f2aa0532939 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:55.302769 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825171b-89f6-4580-bdbb-3f2aa0532939 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:55.304523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 42/90] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:55 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:55.305397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825171b-89f6-4580-bdbb-3f2aa0532939 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:55.334953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29c76b-fb59-41d8-93ce-ad7f5d281b27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:55.336880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29c76b-fb59-41d8-93ce-ad7f5d281b27 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:55.446270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825171b-89f6-4580-bdbb-3f2aa0532939 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:55.461572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 49/91] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:04:55.462410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29c76b-fb59-41d8-93ce-ad7f5d281b27 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:55.492494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:55.493931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:55.493931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:55.493931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:55.494467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=6ca74ec2a86e48a28a003d344862f755, user.id=41ca161a25324736922d4952cd76a1f3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:55.494841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:55.496411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:55.507155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 43/92] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:04:55 2022] GET /identity/v3/role_assignments?user.id=41ca161a25324736922d4952cd76a1f3&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=007af6bd1b734517b934b4cac45faf2c => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:55.508138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:55.523338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:55.566665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:55.636726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc615d-57ad-4076-b0d0-6cb4f1a1aee2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:55.644649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29c76b-fb59-41d8-93ce-ad7f5d281b27 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:55.674621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa792b8-cd0f-44a4-99d5-2217e225362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:55.675527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa792b8-cd0f-44a4-99d5-2217e225362c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:55.676077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa792b8-cd0f-44a4-99d5-2217e225362c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:55.676461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa792b8-cd0f-44a4-99d5-2217e225362c None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:55.677772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa792b8-cd0f-44a4-99d5-2217e225362c None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:55.679287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa792b8-cd0f-44a4-99d5-2217e225362c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:55.681350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 44/93] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:04:55 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:55.687697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa792b8-cd0f-44a4-99d5-2217e225362c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:55.728606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965fa64-cd25-4d53-a30d-dcd3ee5f1fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:55.729280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965fa64-cd25-4d53-a30d-dcd3ee5f1fcd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:55.729565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965fa64-cd25-4d53-a30d-dcd3ee5f1fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:55.729848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965fa64-cd25-4d53-a30d-dcd3ee5f1fcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:55.847976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:55.850651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965fa64-cd25-4d53-a30d-dcd3ee5f1fcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:55.852224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965fa64-cd25-4d53-a30d-dcd3ee5f1fcd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:55.862429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa792b8-cd0f-44a4-99d5-2217e225362c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:55.905546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:55.906782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:55.907290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:55.907857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:55.908889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] RBAC: Adding query filter params (role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=6ca74ec2a86e48a28a003d344862f755, user.id=0d3b8b0ba0b84fecbef6319c3179a020) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:55.909414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:55.910596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:55.912279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:55.918961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 45/94] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:04:55 2022] GET /identity/v3/role_assignments?user.id=0d3b8b0ba0b84fecbef6319c3179a020&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:55.997115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 50/95] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:04:56.003066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:56.152013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c66a6-c4c1-4e72-a271-e5cb9637dda8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:56.189729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df670c7-5122-4e4a-a826-e488d4cdd3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:56.190671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df670c7-5122-4e4a-a826-e488d4cdd3bc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:56.190961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df670c7-5122-4e4a-a826-e488d4cdd3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:56.191241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df670c7-5122-4e4a-a826-e488d4cdd3bc None admin] PATH_INFO: `/v3/projects/a6974ac76aea448a9e2567ecc387b5d8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:56.192540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df670c7-5122-4e4a-a826-e488d4cdd3bc None admin] RBAC: Authorizing `identity:get_project(project_id=a6974ac76aea448a9e2567ecc387b5d8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:56.194017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df670c7-5122-4e4a-a826-e488d4cdd3bc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:56.195809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 46/96] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:04:56 2022] GET /identity/v3/projects/a6974ac76aea448a9e2567ecc387b5d8 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:56.201937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965fa64-cd25-4d53-a30d-dcd3ee5f1fcd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:56.309933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965fa64-cd25-4d53-a30d-dcd3ee5f1fcd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:56.346028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783d7ec-fb84-4ced-b016-8969f9b0e279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:56.346697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d7ec-fb84-4ced-b016-8969f9b0e279 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:56.346983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d7ec-fb84-4ced-b016-8969f9b0e279 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:56.347276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783d7ec-fb84-4ced-b016-8969f9b0e279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:56.348128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783d7ec-fb84-4ced-b016-8969f9b0e279 None admin] RBAC: Adding query filter params (role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=a6974ac76aea448a9e2567ecc387b5d8, user.id=0d3b8b0ba0b84fecbef6319c3179a020) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:56.348499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783d7ec-fb84-4ced-b016-8969f9b0e279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:56.350004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783d7ec-fb84-4ced-b016-8969f9b0e279 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:56.355932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 51/97] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:04:56 2022] GET /identity/v3/role_assignments?user.id=0d3b8b0ba0b84fecbef6319c3179a020&scope.project.id=a6974ac76aea448a9e2567ecc387b5d8&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:57.926411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752672a-110d-4faa-9e37-ca1e9be50eb0 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:57.926928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752672a-110d-4faa-9e37-ca1e9be50eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:57.927002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752672a-110d-4faa-9e37-ca1e9be50eb0 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:57.928434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 47/98] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:57.934052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88cf8a-fe74-4807-8f82-d818887db7c9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:57.934900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88cf8a-fe74-4807-8f82-d818887db7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:57.935404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88cf8a-fe74-4807-8f82-d818887db7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:57.953450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3c88cf8a-fe74-4807-8f82-d818887db7c9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:57.954979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3c88cf8a-fe74-4807-8f82-d818887db7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:57.957193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:58.014999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341ffdd-24f9-46c4-8522-0155a521b653 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.015654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341ffdd-24f9-46c4-8522-0155a521b653 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.016125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341ffdd-24f9-46c4-8522-0155a521b653 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.017934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 48/99] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:58.022604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ec7c4-643d-459c-9fe7-a9d55c013c4a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.023103 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ec7c4-643d-459c-9fe7-a9d55c013c4a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.023660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ec7c4-643d-459c-9fe7-a9d55c013c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.050129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 52/100] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:04:58.055803 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f42ec7c4-643d-459c-9fe7-a9d55c013c4a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:58.057266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f42ec7c4-643d-459c-9fe7-a9d55c013c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:58.075517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:58.097325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88cf8a-fe74-4807-8f82-d818887db7c9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:58.143548 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 49/101] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:04:58.182045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ec7c4-643d-459c-9fe7-a9d55c013c4a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:58.242745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88cf8a-fe74-4807-8f82-d818887db7c9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:58.283698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ec7c4-643d-459c-9fe7-a9d55c013c4a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:58.315982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2580c8f-dbf9-45c8-92a8-da0839572d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:58.316793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2580c8f-dbf9-45c8-92a8-da0839572d41 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.317079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2580c8f-dbf9-45c8-92a8-da0839572d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.317365 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2580c8f-dbf9-45c8-92a8-da0839572d41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.320685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249c13b-b68d-4249-8837-6295e8ddf44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:58.321705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c13b-b68d-4249-8837-6295e8ddf44b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.322182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c13b-b68d-4249-8837-6295e8ddf44b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.322646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249c13b-b68d-4249-8837-6295e8ddf44b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.332616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:58.334258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249c13b-b68d-4249-8837-6295e8ddf44b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:58.336899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2580c8f-dbf9-45c8-92a8-da0839572d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6249c13b-b68d-4249-8837-6295e8ddf44b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:58.343236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:58.347871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:04:58.348652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 50/102] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e2580c8f-dbf9-45c8-92a8-da0839572d41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:58.349293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:58.350048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:04:58.350822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:04:58.351849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed22e1a-b95f-4031-b82b-5ba7b4f7e093 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.351849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed22e1a-b95f-4031-b82b-5ba7b4f7e093 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.351975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 53/103] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:04:58.352048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed22e1a-b95f-4031-b82b-5ba7b4f7e093 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.352048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 51/104] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:58.354981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa8132-56f0-44b6-8296-ab7e318fe640 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.355135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed22e1a-b95f-4031-b82b-5ba7b4f7e093 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:58.355287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa8132-56f0-44b6-8296-ab7e318fe640 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.355287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa8132-56f0-44b6-8296-ab7e318fe640 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.355287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 54/105] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:58.359737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa8132-56f0-44b6-8296-ab7e318fe640 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:58.375638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:58.465685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed22e1a-b95f-4031-b82b-5ba7b4f7e093 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:58.465921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa8132-56f0-44b6-8296-ab7e318fe640 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:58.501508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48cbcb-0d5b-492c-adf5-0e0b3ac941a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:58.501508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48cbcb-0d5b-492c-adf5-0e0b3ac941a0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.501742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48cbcb-0d5b-492c-adf5-0e0b3ac941a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.501808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddb2b4-afce-4678-a845-2dacd6a60bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:58.501917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48cbcb-0d5b-492c-adf5-0e0b3ac941a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.502446 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddb2b4-afce-4678-a845-2dacd6a60bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.502625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48cbcb-0d5b-492c-adf5-0e0b3ac941a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:58.502852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddb2b4-afce-4678-a845-2dacd6a60bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.502947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48cbcb-0d5b-492c-adf5-0e0b3ac941a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:58.503207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddb2b4-afce-4678-a845-2dacd6a60bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.504068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddb2b4-afce-4678-a845-2dacd6a60bf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:58.504457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddb2b4-afce-4678-a845-2dacd6a60bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:58.505518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48cbcb-0d5b-492c-adf5-0e0b3ac941a0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:58.505962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddb2b4-afce-4678-a845-2dacd6a60bf5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:58.515572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 52/106] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:04:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:58.517034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 55/107] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:04:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:58.518310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8a87-6c3a-49a0-ae95-b1d28d6f3805 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.518603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8a87-6c3a-49a0-ae95-b1d28d6f3805 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.518886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8a87-6c3a-49a0-ae95-b1d28d6f3805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.521381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6918-3e36-4a52-8ed4-ca2aaf88bdf9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.521792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6918-3e36-4a52-8ed4-ca2aaf88bdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.522159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec6918-3e36-4a52-8ed4-ca2aaf88bdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.542363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2eec6918-3e36-4a52-8ed4-ca2aaf88bdf9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:58.543709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2eec6918-3e36-4a52-8ed4-ca2aaf88bdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:58.544785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-438b8a87-6c3a-49a0-ae95-b1d28d6f3805 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:58.546610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-438b8a87-6c3a-49a0-ae95-b1d28d6f3805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:58.603420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:58.616495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 56/108] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:58.617281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 53/109] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:58.617665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b8a87-6c3a-49a0-ae95-b1d28d6f3805 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:58.618129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec6918-3e36-4a52-8ed4-ca2aaf88bdf9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:58.621700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:58.724655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b8a87-6c3a-49a0-ae95-b1d28d6f3805 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:58.730371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec6918-3e36-4a52-8ed4-ca2aaf88bdf9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:58.761539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453451e3-6484-42cf-828f-ae08e0f55c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:58.762617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453451e3-6484-42cf-828f-ae08e0f55c00 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.762971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453451e3-6484-42cf-828f-ae08e0f55c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.763347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453451e3-6484-42cf-828f-ae08e0f55c00 None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.765372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453451e3-6484-42cf-828f-ae08e0f55c00 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:58.767091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453451e3-6484-42cf-828f-ae08e0f55c00 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:58.767897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb8e9b-64b8-459d-85fa-0ef454275f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:58.768842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb8e9b-64b8-459d-85fa-0ef454275f4f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:58.768958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 57/110] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:58 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:58.769417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb8e9b-64b8-459d-85fa-0ef454275f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:58.769700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb8e9b-64b8-459d-85fa-0ef454275f4f None admin] PATH_INFO: `/v3/users/d07c08dd1c964eb89b7bc9d1ca53f68f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:58.770002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453451e3-6484-42cf-828f-ae08e0f55c00 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:58.785712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb8e9b-64b8-459d-85fa-0ef454275f4f None admin] RBAC: Authorizing `identity:get_user(user_id=d07c08dd1c964eb89b7bc9d1ca53f68f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:58.787169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb8e9b-64b8-459d-85fa-0ef454275f4f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:58.791326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 54/111] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:58 2022] GET /identity/v3/users/d07c08dd1c964eb89b7bc9d1ca53f68f => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:58.791326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb8e9b-64b8-459d-85fa-0ef454275f4f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:58.923409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:58.930228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:58.967660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb8e9b-64b8-459d-85fa-0ef454275f4f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:58.983993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453451e3-6484-42cf-828f-ae08e0f55c00 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:59.007113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4543-a991-46a2-b393-463a7ec655c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.008271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4543-a991-46a2-b393-463a7ec655c6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.008772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4543-a991-46a2-b393-463a7ec655c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.009355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4543-a991-46a2-b393-463a7ec655c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.020740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c4543-a991-46a2-b393-463a7ec655c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:04:59.025201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37939c46-1938-42b9-bd18-c984eeb677d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.026054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37939c46-1938-42b9-bd18-c984eeb677d3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.026349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37939c46-1938-42b9-bd18-c984eeb677d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.026635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37939c46-1938-42b9-bd18-c984eeb677d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.027883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37939c46-1938-42b9-bd18-c984eeb677d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:59.029894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37939c46-1938-42b9-bd18-c984eeb677d3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-714c4543-a991-46a2-b393-463a7ec655c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:04:59.031235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:04:59.032165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:04:59.032965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:04:59.033575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 58/112] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:04:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:59.033575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37939c46-1938-42b9-bd18-c984eeb677d3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:59.035621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 55/113] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:04:59.036727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4543-a991-46a2-b393-463a7ec655c6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:59.151827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4543-a991-46a2-b393-463a7ec655c6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:59.179689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:59.181386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37939c46-1938-42b9-bd18-c984eeb677d3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:59.194692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e9ef-d03f-434f-b59c-e55c290975f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.199905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e9ef-d03f-434f-b59c-e55c290975f9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.199905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e9ef-d03f-434f-b59c-e55c290975f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.199905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4e9ef-d03f-434f-b59c-e55c290975f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.210993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:59.221112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bda77-4df0-4147-9ab8-daf4287f6e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.222236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bda77-4df0-4147-9ab8-daf4287f6e7d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.222733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bda77-4df0-4147-9ab8-daf4287f6e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.223206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bda77-4df0-4147-9ab8-daf4287f6e7d None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.224642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bda77-4df0-4147-9ab8-daf4287f6e7d None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:59.226295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bda77-4df0-4147-9ab8-daf4287f6e7d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:59.228435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 59/114] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:04:59 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:59.234419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bda77-4df0-4147-9ab8-daf4287f6e7d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:59.277679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e9ef-d03f-434f-b59c-e55c290975f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:59.279305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4e9ef-d03f-434f-b59c-e55c290975f9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:59.344840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bda77-4df0-4147-9ab8-daf4287f6e7d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:59.377950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.378615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.378903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.379185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.379993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:04:59.380312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:59.381025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 56/115] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:04:59.381729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:59.381887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e9ef-d03f-434f-b59c-e55c290975f9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:59.392997 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 60/116] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:59.394011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:59.407900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:59.449195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:59.481554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4e9ef-d03f-434f-b59c-e55c290975f9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:59.493701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e212c5-4227-4ca7-9a67-2a2cb9818dd9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:59.514941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c1809-11ee-4bcf-afec-dfc2d050c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.515630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c1809-11ee-4bcf-afec-dfc2d050c742 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.515918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c1809-11ee-4bcf-afec-dfc2d050c742 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.516198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c1809-11ee-4bcf-afec-dfc2d050c742 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.516914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c1809-11ee-4bcf-afec-dfc2d050c742 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:59.518374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c1809-11ee-4bcf-afec-dfc2d050c742 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:59.525480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f46d91-3e9e-46f0-9004-c4ecc0182319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.526835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46d91-3e9e-46f0-9004-c4ecc0182319 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.527183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46d91-3e9e-46f0-9004-c4ecc0182319 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.527883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46d91-3e9e-46f0-9004-c4ecc0182319 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755/users/0d3b8b0ba0b84fecbef6319c3179a020/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.529941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f46d91-3e9e-46f0-9004-c4ecc0182319 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ca74ec2a86e48a28a003d344862f755, user_id=0d3b8b0ba0b84fecbef6319c3179a020, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:59.531402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f46d91-3e9e-46f0-9004-c4ecc0182319 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:59.648981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 57/117] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:04:59 2022] POST /identity/v3/projects => generated 302 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:04:59.650415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0b786-25b9-4d7f-8873-38883e61c039 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.650720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0b786-25b9-4d7f-8873-38883e61c039 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.651000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0b786-25b9-4d7f-8873-38883e61c039 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.652324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 58/118] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:59.655467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee0ba7-e9f6-482f-9acc-87ef9d391889 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.655467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee0ba7-e9f6-482f-9acc-87ef9d391889 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.655467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee0ba7-e9f6-482f-9acc-87ef9d391889 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.655687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 59/119] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:04:59.656302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee0ba7-e9f6-482f-9acc-87ef9d391889 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:59.663177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 61/120] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:04:59 2022] PUT /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755/users/0d3b8b0ba0b84fecbef6319c3179a020/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:04:59.664405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f46d91-3e9e-46f0-9004-c4ecc0182319 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:59.770708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:59.813279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee0ba7-e9f6-482f-9acc-87ef9d391889 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:59.824125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f46d91-3e9e-46f0-9004-c4ecc0182319 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:04:59.846327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:04:59.861344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9097c4-352b-4201-bfe3-503d0d1fc58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.862442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9097c4-352b-4201-bfe3-503d0d1fc58a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.862732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9097c4-352b-4201-bfe3-503d0d1fc58a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.863003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9097c4-352b-4201-bfe3-503d0d1fc58a None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.864243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9097c4-352b-4201-bfe3-503d0d1fc58a None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:59.865684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9097c4-352b-4201-bfe3-503d0d1fc58a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:59.867763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 60/121] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:04:59 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:59.868865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9014c1d-cdb3-4555-bb89-33cb6c717627 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.869108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9014c1d-cdb3-4555-bb89-33cb6c717627 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.869385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9014c1d-cdb3-4555-bb89-33cb6c717627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.870415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f4528-d336-492a-8c56-288d15c39a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:04:59.871444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4528-d336-492a-8c56-288d15c39a0a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.871824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4528-d336-492a-8c56-288d15c39a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.872201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4528-d336-492a-8c56-288d15c39a0a None admin] PATH_INFO: `/v3/users/41ca161a25324736922d4952cd76a1f3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.874041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f4528-d336-492a-8c56-288d15c39a0a None admin] RBAC: Authorizing `identity:get_user(user_id=41ca161a25324736922d4952cd76a1f3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:04:59.875747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f4528-d336-492a-8c56-288d15c39a0a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:04:59.877671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 62/122] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:04:59 2022] GET /identity/v3/users/41ca161a25324736922d4952cd76a1f3 => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:04:59.878821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950034f7-f3cd-4ee0-ab23-1cb3e8d9f0dd None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:04:59.879531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950034f7-f3cd-4ee0-ab23-1cb3e8d9f0dd None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:04:59.880129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950034f7-f3cd-4ee0-ab23-1cb3e8d9f0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:04:59.892107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b9014c1d-cdb3-4555-bb89-33cb6c717627 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:59.893743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b9014c1d-cdb3-4555-bb89-33cb6c717627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:59.898805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-950034f7-f3cd-4ee0-ab23-1cb3e8d9f0dd None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:04:59.902555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-950034f7-f3cd-4ee0-ab23-1cb3e8d9f0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:04:59.976319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 61/123] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:04:59.977887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9014c1d-cdb3-4555-bb89-33cb6c717627 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:04:59.999592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 63/124] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:00.001236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950034f7-f3cd-4ee0-ab23-1cb3e8d9f0dd None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:00.093736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:00.133944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950034f7-f3cd-4ee0-ab23-1cb3e8d9f0dd None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:00.134523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9014c1d-cdb3-4555-bb89-33cb6c717627 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:00.152474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:00.173105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:00.173953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:00.174405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:00.174826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:00.176207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] RBAC: Adding query filter params (role.id=3582abcd4fa246d4add3f71e7474643f, scope.project.id=6ca74ec2a86e48a28a003d344862f755, user.id=d07c08dd1c964eb89b7bc9d1ca53f68f) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:00.176586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:00.178207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:00.181041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f02046-f104-410a-91aa-59dc52534a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:00.182642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02046-f104-410a-91aa-59dc52534a0f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:00.183348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02046-f104-410a-91aa-59dc52534a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:00.183776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f02046-f104-410a-91aa-59dc52534a0f None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:00.185138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 62/125] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:04:59 2022] GET /identity/v3/role_assignments?user.id=d07c08dd1c964eb89b7bc9d1ca53f68f&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=3582abcd4fa246d4add3f71e7474643f => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:00.186194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f02046-f104-410a-91aa-59dc52534a0f None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:00.186555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:00.188896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f02046-f104-410a-91aa-59dc52534a0f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:00.192474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 64/126] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:00 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:00.197311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f02046-f104-410a-91aa-59dc52534a0f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:00.311431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f02046-f104-410a-91aa-59dc52534a0f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:00.350393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd01713-5dd3-4d7f-900e-18d3a75239bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:00.351601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd01713-5dd3-4d7f-900e-18d3a75239bf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:00.352091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd01713-5dd3-4d7f-900e-18d3a75239bf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:00.352565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd01713-5dd3-4d7f-900e-18d3a75239bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:00.365298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd01713-5dd3-4d7f-900e-18d3a75239bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-edd01713-5dd3-4d7f-900e-18d3a75239bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:00.373618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:00.374567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:00.375404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:00.378321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 65/127] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:00.383518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd01713-5dd3-4d7f-900e-18d3a75239bf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:00.406295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb30ce7-5765-4b6a-935c-72f2912d2e3b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:00.433890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:00.493696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5b942-019f-41b4-b5c6-a9e42f808444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:00.494504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5b942-019f-41b4-b5c6-a9e42f808444 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:00.494788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5b942-019f-41b4-b5c6-a9e42f808444 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:00.495062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5b942-019f-41b4-b5c6-a9e42f808444 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:00.499701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd01713-5dd3-4d7f-900e-18d3a75239bf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:00.502221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:00.521771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5b942-019f-41b4-b5c6-a9e42f808444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-71c5b942-019f-41b4-b5c6-a9e42f808444 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:00.531636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:00.532554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:00.533336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:00.534444 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 63/128] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:00.535141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5b942-019f-41b4-b5c6-a9e42f808444 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:00.555220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e917d7-4933-46a2-99f3-ac9705d47824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:00.556417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e917d7-4933-46a2-99f3-ac9705d47824 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:00.563120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e917d7-4933-46a2-99f3-ac9705d47824 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:00.563120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e917d7-4933-46a2-99f3-ac9705d47824 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755/users/41ca161a25324736922d4952cd76a1f3/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:00.563120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e917d7-4933-46a2-99f3-ac9705d47824 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ca74ec2a86e48a28a003d344862f755, user_id=41ca161a25324736922d4952cd76a1f3, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:00.563120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e917d7-4933-46a2-99f3-ac9705d47824 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:00.666197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 66/129] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:00 2022] PUT /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755/users/41ca161a25324736922d4952cd76a1f3/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:00.666897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e917d7-4933-46a2-99f3-ac9705d47824 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:00.674471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5b942-019f-41b4-b5c6-a9e42f808444 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:00.757518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:00.758217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:00.758510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:00.758798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:00.759598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:00.759915 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:00.761321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:00.781130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 64/130] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:00.789627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:00.841185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e917d7-4933-46a2-99f3-ac9705d47824 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:00.845766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:00.892542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:00.930529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:00.930529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:00.930529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:00.930529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:00.930937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:00.931031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:00.932452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:00.940248 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995b51d-d87b-4ff6-9421-9d01af8b52bc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:00.950944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 67/131] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:00.971132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:00.984170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe570f-2fd6-4209-b557-eefd7a1b3a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:00.985061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe570f-2fd6-4209-b557-eefd7a1b3a07 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:00.985346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe570f-2fd6-4209-b557-eefd7a1b3a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:00.985626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe570f-2fd6-4209-b557-eefd7a1b3a07 None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:00.986903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe570f-2fd6-4209-b557-eefd7a1b3a07 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:00.988407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe570f-2fd6-4209-b557-eefd7a1b3a07 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:00.990158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 65/132] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:00 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:00.995740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe570f-2fd6-4209-b557-eefd7a1b3a07 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:01.104186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe570f-2fd6-4209-b557-eefd7a1b3a07 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:01.131484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:01.141233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d274-24bc-4389-bc07-ee164ccb2dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:01.142167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d274-24bc-4389-bc07-ee164ccb2dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:01.142463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d274-24bc-4389-bc07-ee164ccb2dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:01.142744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6d274-24bc-4389-bc07-ee164ccb2dd4 None admin] PATH_INFO: `/v3/projects/a6974ac76aea448a9e2567ecc387b5d8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:01.143972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6d274-24bc-4389-bc07-ee164ccb2dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=a6974ac76aea448a9e2567ecc387b5d8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:01.145421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6d274-24bc-4389-bc07-ee164ccb2dd4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:01.151392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 66/133] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:00 2022] GET /identity/v3/projects/a6974ac76aea448a9e2567ecc387b5d8 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:01.152837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d274-24bc-4389-bc07-ee164ccb2dd4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:01.199813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355cec4-c709-463b-9372-47abdde17d81 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:01.258060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6d274-24bc-4389-bc07-ee164ccb2dd4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:01.267940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800205e-5f51-4a79-9d21-dbcb17ab7835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:01.268832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800205e-5f51-4a79-9d21-dbcb17ab7835 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:01.269127 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800205e-5f51-4a79-9d21-dbcb17ab7835 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:01.269413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800205e-5f51-4a79-9d21-dbcb17ab7835 None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:01.270688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800205e-5f51-4a79-9d21-dbcb17ab7835 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:01.272179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800205e-5f51-4a79-9d21-dbcb17ab7835 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:01.273837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 68/134] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:00 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:01.298857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6efc1f-d035-453e-8b6e-3cd2f64dcd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:01.299825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6efc1f-d035-453e-8b6e-3cd2f64dcd58 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:01.300111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6efc1f-d035-453e-8b6e-3cd2f64dcd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:01.300276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800205e-5f51-4a79-9d21-dbcb17ab7835 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:01.300443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6efc1f-d035-453e-8b6e-3cd2f64dcd58 None admin] PATH_INFO: `/v3/projects/a6974ac76aea448a9e2567ecc387b5d8/users/0d3b8b0ba0b84fecbef6319c3179a020/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:01.302763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6efc1f-d035-453e-8b6e-3cd2f64dcd58 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6974ac76aea448a9e2567ecc387b5d8, user_id=0d3b8b0ba0b84fecbef6319c3179a020, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:01.304327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6efc1f-d035-453e-8b6e-3cd2f64dcd58 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:01.369627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:01.402560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 67/135] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:01 2022] PUT /identity/v3/projects/a6974ac76aea448a9e2567ecc387b5d8/users/0d3b8b0ba0b84fecbef6319c3179a020/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:01.464151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800205e-5f51-4a79-9d21-dbcb17ab7835 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:01.505956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df531d4-ef32-44e1-8e6d-169e225d18f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:01.506859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df531d4-ef32-44e1-8e6d-169e225d18f4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:01.507149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df531d4-ef32-44e1-8e6d-169e225d18f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:01.507465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df531d4-ef32-44e1-8e6d-169e225d18f4 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:01.508832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df531d4-ef32-44e1-8e6d-169e225d18f4 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:01.510340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df531d4-ef32-44e1-8e6d-169e225d18f4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:01.512217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 69/136] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:01 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:01.518258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6efc1f-d035-453e-8b6e-3cd2f64dcd58 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:01.608214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:01.668912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6efc1f-d035-453e-8b6e-3cd2f64dcd58 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:01.710157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611be4c-7792-4c66-95af-c882730c6727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:01.711051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611be4c-7792-4c66-95af-c882730c6727 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:01.711359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611be4c-7792-4c66-95af-c882730c6727 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:01.711638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611be4c-7792-4c66-95af-c882730c6727 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755/users/0d3b8b0ba0b84fecbef6319c3179a020/roles/9eaabfe45189403f8cee9512fefbba65` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:01.713924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611be4c-7792-4c66-95af-c882730c6727 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ca74ec2a86e48a28a003d344862f755, user_id=0d3b8b0ba0b84fecbef6319c3179a020, role_id=9eaabfe45189403f8cee9512fefbba65)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:01.715455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611be4c-7792-4c66-95af-c882730c6727 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:01.801243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 68/137] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:01 2022] PUT /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755/users/0d3b8b0ba0b84fecbef6319c3179a020/roles/9eaabfe45189403f8cee9512fefbba65 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:02.349869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2614a-fb70-4f08-9903-e2042dacf026 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:02.350362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2614a-fb70-4f08-9903-e2042dacf026 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:02.350362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba2614a-fb70-4f08-9903-e2042dacf026 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:02.351767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 70/138] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:02.357003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb3115-efc3-4574-8458-c6d78e2bd830 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:02.357969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb3115-efc3-4574-8458-c6d78e2bd830 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:02.358110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb3115-efc3-4574-8458-c6d78e2bd830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:02.375352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2eeb3115-efc3-4574-8458-c6d78e2bd830 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:02.376571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2eeb3115-efc3-4574-8458-c6d78e2bd830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:02.447179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 69/139] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:02.470504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2614a-fb70-4f08-9903-e2042dacf026 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:02.602990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:02.611490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba2614a-fb70-4f08-9903-e2042dacf026 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:02.646567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82311d1b-8e0e-4d8c-92f8-e8ec87c07b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:02.647402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82311d1b-8e0e-4d8c-92f8-e8ec87c07b8d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:02.647703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82311d1b-8e0e-4d8c-92f8-e8ec87c07b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:02.647991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82311d1b-8e0e-4d8c-92f8-e8ec87c07b8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:02.658752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82311d1b-8e0e-4d8c-92f8-e8ec87c07b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-82311d1b-8e0e-4d8c-92f8-e8ec87c07b8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:02.666491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:02.667466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:02.668246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 71/140] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:02.674279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb3115-efc3-4574-8458-c6d78e2bd830 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:02.724877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae30e-dd30-4d15-891d-8e90ed0339eb None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:02.725505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae30e-dd30-4d15-891d-8e90ed0339eb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:02.726002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bae30e-dd30-4d15-891d-8e90ed0339eb None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:02.727725 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 72/141] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:02.731733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0ab7d-5f40-4c05-b3ee-5dbe791fc233 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:02.732318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0ab7d-5f40-4c05-b3ee-5dbe791fc233 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:02.733287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0ab7d-5f40-4c05-b3ee-5dbe791fc233 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:02.734790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 73/142] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:02.739197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97a710-7a9b-4a22-a108-a1cc8813c95b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:02.741760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97a710-7a9b-4a22-a108-a1cc8813c95b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:02.741760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97a710-7a9b-4a22-a108-a1cc8813c95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:02.744594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:02.775461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ba97a710-7a9b-4a22-a108-a1cc8813c95b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:02.776688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ba97a710-7a9b-4a22-a108-a1cc8813c95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:02.803294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb3115-efc3-4574-8458-c6d78e2bd830 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:02.851575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bae76-22c3-40d5-9907-b364296b3b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:02.852342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bae76-22c3-40d5-9907-b364296b3b86 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:02.852637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bae76-22c3-40d5-9907-b364296b3b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:02.852922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bae76-22c3-40d5-9907-b364296b3b86 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:02.853742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bae76-22c3-40d5-9907-b364296b3b86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:02.854002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bae76-22c3-40d5-9907-b364296b3b86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:02.855435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bae76-22c3-40d5-9907-b364296b3b86 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:02.866410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 70/143] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:02.868245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572195-eeb4-468b-b93d-eed59a729e56 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:02.869041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572195-eeb4-468b-b93d-eed59a729e56 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:02.869331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70572195-eeb4-468b-b93d-eed59a729e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:02.889116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-70572195-eeb4-468b-b93d-eed59a729e56 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:02.890330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-70572195-eeb4-468b-b93d-eed59a729e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:02.932899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 74/144] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:02.933824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97a710-7a9b-4a22-a108-a1cc8813c95b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:02.961719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 71/145] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:02.973748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572195-eeb4-468b-b93d-eed59a729e56 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:03.038913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:03.109346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:03.110997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70572195-eeb4-468b-b93d-eed59a729e56 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:03.142055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97a710-7a9b-4a22-a108-a1cc8813c95b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:03.147591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f4406-c7f4-4715-812b-5ba5cf942434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:03.148520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f4406-c7f4-4715-812b-5ba5cf942434 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:03.148840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f4406-c7f4-4715-812b-5ba5cf942434 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:03.149072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f4406-c7f4-4715-812b-5ba5cf942434 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:03.150307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f4406-c7f4-4715-812b-5ba5cf942434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:03.151839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f4406-c7f4-4715-812b-5ba5cf942434 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:03.153590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 72/146] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:03.154443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f4406-c7f4-4715-812b-5ba5cf942434 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:03.235973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c9197-6adc-4a31-8993-c034aa1134d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:03.236872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9197-6adc-4a31-8993-c034aa1134d0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:03.237162 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9197-6adc-4a31-8993-c034aa1134d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:03.237443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c9197-6adc-4a31-8993-c034aa1134d0 None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:03.238711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c9197-6adc-4a31-8993-c034aa1134d0 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:03.240183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c9197-6adc-4a31-8993-c034aa1134d0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:03.241843 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 75/147] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:02 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:03.246901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c9197-6adc-4a31-8993-c034aa1134d0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:03.260523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f4406-c7f4-4715-812b-5ba5cf942434 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:03.263113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:03.297241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a04e8-8509-400b-a7c8-df120bd7097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:03.298094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a04e8-8509-400b-a7c8-df120bd7097f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:03.298375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a04e8-8509-400b-a7c8-df120bd7097f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:03.298694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a04e8-8509-400b-a7c8-df120bd7097f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:03.310903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a04e8-8509-400b-a7c8-df120bd7097f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-197a04e8-8509-400b-a7c8-df120bd7097f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:03.318873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:03.319793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:03.320566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 73/148] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:03.321211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a04e8-8509-400b-a7c8-df120bd7097f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:03.415161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c9197-6adc-4a31-8993-c034aa1134d0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:03.460340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed69ea-c7f0-4137-94e5-5d23c719ce48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:03.462164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed69ea-c7f0-4137-94e5-5d23c719ce48 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:03.462445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed69ea-c7f0-4137-94e5-5d23c719ce48 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:03.462720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed69ea-c7f0-4137-94e5-5d23c719ce48 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:03.465473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed69ea-c7f0-4137-94e5-5d23c719ce48 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:03.467472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed69ea-c7f0-4137-94e5-5d23c719ce48 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:03.493924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a04e8-8509-400b-a7c8-df120bd7097f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:03.531800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbf3e4-53cc-4067-8a4c-1fdaff889679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:03.532454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf3e4-53cc-4067-8a4c-1fdaff889679 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:03.532744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf3e4-53cc-4067-8a4c-1fdaff889679 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:03.533115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf3e4-53cc-4067-8a4c-1fdaff889679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:03.566925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:03.575086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 76/149] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:03 2022] POST /identity/v3/users => generated 283 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:03.576708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed69ea-c7f0-4137-94e5-5d23c719ce48 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:03.612145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:03.640778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbf3e4-53cc-4067-8a4c-1fdaff889679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:03.642319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbf3e4-53cc-4067-8a4c-1fdaff889679 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:03.761354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 74/150] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:03.761678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed69ea-c7f0-4137-94e5-5d23c719ce48 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:03.772660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbf3e4-53cc-4067-8a4c-1fdaff889679 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:03.842732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:03.843438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:03.846137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:03.846426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:03.847212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:03.851586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:03.853073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:03.869685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 77/151] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:03.880803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:03.900354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:03.972854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbf3e4-53cc-4067-8a4c-1fdaff889679 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.025874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:04.031363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea5ed2-98b8-4501-9546-2e49aee64dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:04.032549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea5ed2-98b8-4501-9546-2e49aee64dbc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.033606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea5ed2-98b8-4501-9546-2e49aee64dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.034189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea5ed2-98b8-4501-9546-2e49aee64dbc None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.035662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea5ed2-98b8-4501-9546-2e49aee64dbc None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:04.037278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea5ed2-98b8-4501-9546-2e49aee64dbc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:04.045409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 75/152] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:03 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:04.045409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413915d-523f-4da8-a9c1-5b134c5fe067 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.045409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413915d-523f-4da8-a9c1-5b134c5fe067 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.045409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413915d-523f-4da8-a9c1-5b134c5fe067 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.045409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 76/153] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:04.047453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413915d-523f-4da8-a9c1-5b134c5fe067 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:04.061248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9eb25-ef0f-4bf5-9574-4fcff2df8d08 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.105918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641bc3a-cde3-4255-894d-63f46ae0faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:04.106112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641bc3a-cde3-4255-894d-63f46ae0faa3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.106351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641bc3a-cde3-4255-894d-63f46ae0faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.106530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641bc3a-cde3-4255-894d-63f46ae0faa3 None admin] PATH_INFO: `/v3/users/d07c08dd1c964eb89b7bc9d1ca53f68f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.108430 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641bc3a-cde3-4255-894d-63f46ae0faa3 None admin] RBAC: Authorizing `identity:get_user(user_id=d07c08dd1c964eb89b7bc9d1ca53f68f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:04.110116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641bc3a-cde3-4255-894d-63f46ae0faa3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:04.111796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 78/154] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:03 2022] GET /identity/v3/users/d07c08dd1c964eb89b7bc9d1ca53f68f => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:04.112946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf74a4-d548-4d55-a91f-6a12cd9e365b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.113231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf74a4-d548-4d55-a91f-6a12cd9e365b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.113506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf74a4-d548-4d55-a91f-6a12cd9e365b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.131959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-afaf74a4-d548-4d55-a91f-6a12cd9e365b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:04.134454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-afaf74a4-d548-4d55-a91f-6a12cd9e365b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:04.170496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413915d-523f-4da8-a9c1-5b134c5fe067 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.209439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:04.219979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 79/155] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:04.220691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966dd8c-8fe5-47a7-8a7c-4f39305624e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:04.223790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf74a4-d548-4d55-a91f-6a12cd9e365b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:04.223901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966dd8c-8fe5-47a7-8a7c-4f39305624e0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.223901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966dd8c-8fe5-47a7-8a7c-4f39305624e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.223901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966dd8c-8fe5-47a7-8a7c-4f39305624e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.223901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966dd8c-8fe5-47a7-8a7c-4f39305624e0 None admin] RBAC: Adding query filter params (role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=6ca74ec2a86e48a28a003d344862f755, user.id=0d3b8b0ba0b84fecbef6319c3179a020) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:04.223901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966dd8c-8fe5-47a7-8a7c-4f39305624e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:04.224712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966dd8c-8fe5-47a7-8a7c-4f39305624e0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:04.231778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 77/156] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:04 2022] GET /identity/v3/role_assignments?user.id=0d3b8b0ba0b84fecbef6319c3179a020&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:04.232931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d6b78-befa-4305-9af6-b57113e053ad None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.233221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d6b78-befa-4305-9af6-b57113e053ad None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.233495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d6b78-befa-4305-9af6-b57113e053ad None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.234723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 78/157] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:04.245238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510606e-1dff-4475-a35f-35b5df4c21f1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.245560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510606e-1dff-4475-a35f-35b5df4c21f1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.245799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510606e-1dff-4475-a35f-35b5df4c21f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.283607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1510606e-1dff-4475-a35f-35b5df4c21f1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:04.285286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1510606e-1dff-4475-a35f-35b5df4c21f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:04.339138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:04.376218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf74a4-d548-4d55-a91f-6a12cd9e365b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.395711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 79/158] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:04.397983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510606e-1dff-4475-a35f-35b5df4c21f1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:04.430845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7077cf-b0b6-4ef5-a1cd-08c369d339da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:04.431859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7077cf-b0b6-4ef5-a1cd-08c369d339da None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.432125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7077cf-b0b6-4ef5-a1cd-08c369d339da None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.432404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7077cf-b0b6-4ef5-a1cd-08c369d339da None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.433645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7077cf-b0b6-4ef5-a1cd-08c369d339da None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:04.435166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7077cf-b0b6-4ef5-a1cd-08c369d339da None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:04.437792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 80/159] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:04 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:04.438654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7077cf-b0b6-4ef5-a1cd-08c369d339da None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:04.520007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510606e-1dff-4475-a35f-35b5df4c21f1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.544937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7077cf-b0b6-4ef5-a1cd-08c369d339da None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.548764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:04.558182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff607cc-160d-4e1b-ade0-3985e6f75c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:04.559294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff607cc-160d-4e1b-ade0-3985e6f75c35 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.559739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff607cc-160d-4e1b-ade0-3985e6f75c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.560412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff607cc-160d-4e1b-ade0-3985e6f75c35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.572795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff607cc-160d-4e1b-ade0-3985e6f75c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-aff607cc-160d-4e1b-ade0-3985e6f75c35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:04.582704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:04.583666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:04.584530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:04.585119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea775130-5cc2-4570-b09a-7824a3987f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:04.585119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea775130-5cc2-4570-b09a-7824a3987f91 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.585119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea775130-5cc2-4570-b09a-7824a3987f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.585119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea775130-5cc2-4570-b09a-7824a3987f91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.586915 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 80/160] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:04.587872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff607cc-160d-4e1b-ade0-3985e6f75c35 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:04.601689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea775130-5cc2-4570-b09a-7824a3987f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ea775130-5cc2-4570-b09a-7824a3987f91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:04.611201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:04.612139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:04.612940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 81/161] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:04.613610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea775130-5cc2-4570-b09a-7824a3987f91 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:04.616551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:04.736816 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea775130-5cc2-4570-b09a-7824a3987f91 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.753701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff607cc-160d-4e1b-ade0-3985e6f75c35 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.777576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:04.778252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.778541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.778824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.779704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:04.780019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:04.781638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:04.791466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 82/162] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:04.793020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:04.798613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a5d2b-42a8-4124-a260-868a67dec069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:04.799809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5d2b-42a8-4124-a260-868a67dec069 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:04.800291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5d2b-42a8-4124-a260-868a67dec069 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:04.801204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a5d2b-42a8-4124-a260-868a67dec069 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755/users/d07c08dd1c964eb89b7bc9d1ca53f68f/roles/3582abcd4fa246d4add3f71e7474643f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:04.804353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a5d2b-42a8-4124-a260-868a67dec069 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ca74ec2a86e48a28a003d344862f755, user_id=d07c08dd1c964eb89b7bc9d1ca53f68f, role_id=3582abcd4fa246d4add3f71e7474643f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:04.806059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a5d2b-42a8-4124-a260-868a67dec069 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:04.871916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:04.888958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 81/163] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:04 2022] PUT /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755/users/d07c08dd1c964eb89b7bc9d1ca53f68f/roles/3582abcd4fa246d4add3f71e7474643f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:04.889885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a5d2b-42a8-4124-a260-868a67dec069 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:04.943630 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386ce63-1d31-4a03-ac7c-35deb718dd34 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:04.997628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:05.026036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:05.026735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.027017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.027307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.028074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:05.028382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:05.030359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:05.048342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 83/164] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:05.056693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:05.139372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a5d2b-42a8-4124-a260-868a67dec069 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:05.182491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18fcd4-5bc2-46d8-86eb-f0a36a54464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:05.183411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18fcd4-5bc2-46d8-86eb-f0a36a54464b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.183692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18fcd4-5bc2-46d8-86eb-f0a36a54464b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.183965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18fcd4-5bc2-46d8-86eb-f0a36a54464b None admin] PATH_INFO: `/v3/users/41ca161a25324736922d4952cd76a1f3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.185808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18fcd4-5bc2-46d8-86eb-f0a36a54464b None admin] RBAC: Authorizing `identity:get_user(user_id=41ca161a25324736922d4952cd76a1f3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:05.187416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18fcd4-5bc2-46d8-86eb-f0a36a54464b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:05.189408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 82/165] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:04 2022] GET /identity/v3/users/41ca161a25324736922d4952cd76a1f3 => generated 247 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:05.190681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9d6c8-33ef-4f9a-a66a-e8925300d18e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.190968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9d6c8-33ef-4f9a-a66a-e8925300d18e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.191240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9d6c8-33ef-4f9a-a66a-e8925300d18e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.193039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 83/166] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:05.197899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9d6c8-33ef-4f9a-a66a-e8925300d18e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:05.209893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc6df-ff61-43be-82ed-b4e07286d385 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:05.271043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:05.288859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa0d9-2e38-4956-9760-af3d773af11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:05.290128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa0d9-2e38-4956-9760-af3d773af11e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.290688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa0d9-2e38-4956-9760-af3d773af11e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.291182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa0d9-2e38-4956-9760-af3d773af11e None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.295444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962aa0d9-2e38-4956-9760-af3d773af11e None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:05.297216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962aa0d9-2e38-4956-9760-af3d773af11e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:05.299128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 84/167] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:05 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:05.302110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4438334-52d6-47ef-8d2f-9ef725b31dce None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.302682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4438334-52d6-47ef-8d2f-9ef725b31dce None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.303156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4438334-52d6-47ef-8d2f-9ef725b31dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.310793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9d6c8-33ef-4f9a-a66a-e8925300d18e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:05.325655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:05.330962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a4438334-52d6-47ef-8d2f-9ef725b31dce None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:05.332410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a4438334-52d6-47ef-8d2f-9ef725b31dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:05.347592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139ad1-3fd7-41bd-bd26-1d09ab46d08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:05.348255 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139ad1-3fd7-41bd-bd26-1d09ab46d08c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.348530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139ad1-3fd7-41bd-bd26-1d09ab46d08c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.348802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139ad1-3fd7-41bd-bd26-1d09ab46d08c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.422796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb139ad1-3fd7-41bd-bd26-1d09ab46d08c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:05.424283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb139ad1-3fd7-41bd-bd26-1d09ab46d08c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:05.441705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 85/168] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:05.442366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4438334-52d6-47ef-8d2f-9ef725b31dce None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:05.558156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 84/169] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:05.559133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139ad1-3fd7-41bd-bd26-1d09ab46d08c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:05.568587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4438334-52d6-47ef-8d2f-9ef725b31dce None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:05.574863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:05.593073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:05.603716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bf59d-11a8-4629-9285-962dde804154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:05.604369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bf59d-11a8-4629-9285-962dde804154 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.604653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bf59d-11a8-4629-9285-962dde804154 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.604930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bf59d-11a8-4629-9285-962dde804154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.669168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139ad1-3fd7-41bd-bd26-1d09ab46d08c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:05.674891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902bf59d-11a8-4629-9285-962dde804154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:05.676690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902bf59d-11a8-4629-9285-962dde804154 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:05.707451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b50711-1b95-42e8-a7b7-5f9074f6d48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:05.708534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b50711-1b95-42e8-a7b7-5f9074f6d48d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.708973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b50711-1b95-42e8-a7b7-5f9074f6d48d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.709389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b50711-1b95-42e8-a7b7-5f9074f6d48d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.721251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b50711-1b95-42e8-a7b7-5f9074f6d48d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-58b50711-1b95-42e8-a7b7-5f9074f6d48d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:05.730456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:05.731596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:05.732636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:05.735131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 85/170] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:05.740574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b50711-1b95-42e8-a7b7-5f9074f6d48d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:05.778850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 86/171] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:05.780617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bf59d-11a8-4629-9285-962dde804154 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:05.806346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:05.828426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:05.856791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b50711-1b95-42e8-a7b7-5f9074f6d48d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:05.884836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bf59d-11a8-4629-9285-962dde804154 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:05.894502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486df19e-264d-48aa-ae79-94710f24ace6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:05.895710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486df19e-264d-48aa-ae79-94710f24ace6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.896220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486df19e-264d-48aa-ae79-94710f24ace6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.896668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486df19e-264d-48aa-ae79-94710f24ace6 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.898045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486df19e-264d-48aa-ae79-94710f24ace6 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:05.899971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486df19e-264d-48aa-ae79-94710f24ace6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:05.901996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 86/172] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:05 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:05.903024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486df19e-264d-48aa-ae79-94710f24ace6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:05.920790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:05.921265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:05.921554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:05.921836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:05.922590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:05.922899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:05.924357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:05.932977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 87/173] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:05.933797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:06.020592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:06.047840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adc167-70cb-475a-bf5f-2911e8f14fc7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:06.079340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486df19e-264d-48aa-ae79-94710f24ace6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:06.093526 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:06.094192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:06.094475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:06.094750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:06.095623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=6ca74ec2a86e48a28a003d344862f755, user.id=41ca161a25324736922d4952cd76a1f3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:06.095938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:06.097398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:06.107834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 88/174] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:05 2022] GET /identity/v3/role_assignments?user.id=41ca161a25324736922d4952cd76a1f3&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=007af6bd1b734517b934b4cac45faf2c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:06.108461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:06.168627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae776f00-3cec-4244-96f9-978068019a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:06.168627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae776f00-3cec-4244-96f9-978068019a38 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:06.168627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae776f00-3cec-4244-96f9-978068019a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:06.168627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae776f00-3cec-4244-96f9-978068019a38 None admin] PATH_INFO: `/v3/projects/a6974ac76aea448a9e2567ecc387b5d8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:06.170082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae776f00-3cec-4244-96f9-978068019a38 None admin] RBAC: Authorizing `identity:get_project(project_id=a6974ac76aea448a9e2567ecc387b5d8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:06.171604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae776f00-3cec-4244-96f9-978068019a38 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:06.175852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 87/175] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:05 2022] GET /identity/v3/projects/a6974ac76aea448a9e2567ecc387b5d8 => generated 312 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:06.179810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae776f00-3cec-4244-96f9-978068019a38 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:06.196570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:06.292613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f596ca-3498-4aba-a85b-86181fbf1c92 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:06.376670 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae776f00-3cec-4244-96f9-978068019a38 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:06.379033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8207c8-c051-4895-b8c6-00d4a7896811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:06.379943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8207c8-c051-4895-b8c6-00d4a7896811 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:06.380226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8207c8-c051-4895-b8c6-00d4a7896811 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:06.380502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8207c8-c051-4895-b8c6-00d4a7896811 None admin] PATH_INFO: `/v3/users/0d3b8b0ba0b84fecbef6319c3179a020` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:06.381805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8207c8-c051-4895-b8c6-00d4a7896811 None admin] RBAC: Authorizing `identity:get_user(user_id=0d3b8b0ba0b84fecbef6319c3179a020)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:06.384227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8207c8-c051-4895-b8c6-00d4a7896811 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:06.385884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 89/176] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:06 2022] GET /identity/v3/users/0d3b8b0ba0b84fecbef6319c3179a020 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:06.392356 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8207c8-c051-4895-b8c6-00d4a7896811 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:06.439429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:06.447377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:06.450544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:06.450544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:06.450544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:06.450544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] RBAC: Adding query filter params (role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=a6974ac76aea448a9e2567ecc387b5d8, user.id=0d3b8b0ba0b84fecbef6319c3179a020) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:06.450544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:06.452167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:06.458733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 88/177] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:06 2022] GET /identity/v3/role_assignments?user.id=0d3b8b0ba0b84fecbef6319c3179a020&scope.project.id=a6974ac76aea448a9e2567ecc387b5d8&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:06.574850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8207c8-c051-4895-b8c6-00d4a7896811 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:06.617618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2d7d0-87f5-4b43-b312-a0b58d01206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:06.618271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d7d0-87f5-4b43-b312-a0b58d01206d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:06.618555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d7d0-87f5-4b43-b312-a0b58d01206d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:06.618832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2d7d0-87f5-4b43-b312-a0b58d01206d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:06.698095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2d7d0-87f5-4b43-b312-a0b58d01206d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:06.699699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2d7d0-87f5-4b43-b312-a0b58d01206d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:06.722116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:06.827736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 90/178] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:06.833967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:06.926056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:06.951280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a105e0-a26c-4c30-a5bc-a43ef7ce54d5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:06.995451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6042f8a-0596-4c59-a958-7a5e79903848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:06.996323 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6042f8a-0596-4c59-a958-7a5e79903848 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:06.996617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6042f8a-0596-4c59-a958-7a5e79903848 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:06.996890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6042f8a-0596-4c59-a958-7a5e79903848 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:06.998161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6042f8a-0596-4c59-a958-7a5e79903848 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:06.999623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6042f8a-0596-4c59-a958-7a5e79903848 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:07.001640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 89/179] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:06 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:07.007384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2d7d0-87f5-4b43-b312-a0b58d01206d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:07.140454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2d7d0-87f5-4b43-b312-a0b58d01206d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:07.159355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:07.178289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8966c-7b96-44d1-9a66-4ec81e18fac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:07.178954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8966c-7b96-44d1-9a66-4ec81e18fac1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.179240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8966c-7b96-44d1-9a66-4ec81e18fac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.179545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8966c-7b96-44d1-9a66-4ec81e18fac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.180389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8966c-7b96-44d1-9a66-4ec81e18fac1 None admin] RBAC: Adding query filter params (role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=6ca74ec2a86e48a28a003d344862f755, user.id=0d3b8b0ba0b84fecbef6319c3179a020) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:07.180702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8966c-7b96-44d1-9a66-4ec81e18fac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:07.182147 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8966c-7b96-44d1-9a66-4ec81e18fac1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:07.188872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 91/180] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:07 2022] GET /identity/v3/role_assignments?user.id=0d3b8b0ba0b84fecbef6319c3179a020&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:07.469532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ef573-9663-400a-a9ad-d3ec9404ce66 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.469896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ef573-9663-400a-a9ad-d3ec9404ce66 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.470097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ef573-9663-400a-a9ad-d3ec9404ce66 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.471512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 90/181] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:07.477126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea69f4-6eab-4981-97de-a5febdbff73e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.477422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea69f4-6eab-4981-97de-a5febdbff73e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.477705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea69f4-6eab-4981-97de-a5febdbff73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.496920 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-85ea69f4-6eab-4981-97de-a5febdbff73e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:07.498708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-85ea69f4-6eab-4981-97de-a5febdbff73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:07.581007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 92/182] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:07.604179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ef573-9663-400a-a9ad-d3ec9404ce66 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:07.663523 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6029c-8a00-4a34-a889-68f1abc96933 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.663823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6029c-8a00-4a34-a889-68f1abc96933 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.664086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6029c-8a00-4a34-a889-68f1abc96933 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.665604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 93/183] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:07.672196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70ff1e-d685-42f3-98f4-c0cfd0d5017f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.672483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70ff1e-d685-42f3-98f4-c0cfd0d5017f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.672767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70ff1e-d685-42f3-98f4-c0cfd0d5017f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.694186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6c70ff1e-d685-42f3-98f4-c0cfd0d5017f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:07.695501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6c70ff1e-d685-42f3-98f4-c0cfd0d5017f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:07.726676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ef573-9663-400a-a9ad-d3ec9404ce66 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:07.764583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92052-cb92-4aa2-be7b-7df45821c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:07.765383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92052-cb92-4aa2-be7b-7df45821c8dd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.765667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92052-cb92-4aa2-be7b-7df45821c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.765943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b92052-cb92-4aa2-be7b-7df45821c8dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.771376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 94/184] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:07.776782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b92052-cb92-4aa2-be7b-7df45821c8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-64b92052-cb92-4aa2-be7b-7df45821c8dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:07.784592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:07.785500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:07.786295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 91/185] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:07.791652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70ff1e-d685-42f3-98f4-c0cfd0d5017f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:07.794206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92052-cb92-4aa2-be7b-7df45821c8dd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:07.799563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:07.849348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:07.915795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b92052-cb92-4aa2-be7b-7df45821c8dd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:07.923668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70ff1e-d685-42f3-98f4-c0cfd0d5017f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:07.949093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622613c-5ff6-4d11-ac5c-39d021aa7e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:07.949910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622613c-5ff6-4d11-ac5c-39d021aa7e25 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.950180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622613c-5ff6-4d11-ac5c-39d021aa7e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.950466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622613c-5ff6-4d11-ac5c-39d021aa7e25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.960902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622613c-5ff6-4d11-ac5c-39d021aa7e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:07.964930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9ada4-8b0d-420e-bb1a-28be7f6c4e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:07.965595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ada4-8b0d-420e-bb1a-28be7f6c4e05 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.965879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ada4-8b0d-420e-bb1a-28be7f6c4e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.966158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ada4-8b0d-420e-bb1a-28be7f6c4e05 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.966942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9ada4-8b0d-420e-bb1a-28be7f6c4e05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:07.967272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9ada4-8b0d-420e-bb1a-28be7f6c4e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:07.968656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9ada4-8b0d-420e-bb1a-28be7f6c4e05 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-b622613c-5ff6-4d11-ac5c-39d021aa7e25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:07.968884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:07.969822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:07.971508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 92/186] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:07.972177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9673b5-cae3-4aac-b55d-6f4a227b6172 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.972177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9673b5-cae3-4aac-b55d-6f4a227b6172 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.972177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9673b5-cae3-4aac-b55d-6f4a227b6172 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:07.973523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 93/187] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:07.975712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9673b5-cae3-4aac-b55d-6f4a227b6172 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:07.987953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 95/188] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:07.987953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affaba-3684-4635-b999-7ace14fc35d1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:07.987953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affaba-3684-4635-b999-7ace14fc35d1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:07.987953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5affaba-3684-4635-b999-7ace14fc35d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.004060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d5affaba-3684-4635-b999-7ace14fc35d1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:08.005241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d5affaba-3684-4635-b999-7ace14fc35d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:08.050217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.086808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 96/189] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:08.087490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5affaba-3684-4635-b999-7ace14fc35d1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:08.106435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9673b5-cae3-4aac-b55d-6f4a227b6172 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:08.127713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.143847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:08.144629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.144916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.145215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.145969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:08.146322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:08.147721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:08.156381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 94/190] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:08.157192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:08.203078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5affaba-3684-4635-b999-7ace14fc35d1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:08.237096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6cfbf-f4e7-4e0d-85d9-a1fd214fe02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:08.238016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cfbf-f4e7-4e0d-85d9-a1fd214fe02f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.238243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cfbf-f4e7-4e0d-85d9-a1fd214fe02f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.238513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6cfbf-f4e7-4e0d-85d9-a1fd214fe02f None admin] PATH_INFO: `/v3/users/e4316a12a39f42a08812c00e07f343a0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.251317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6cfbf-f4e7-4e0d-85d9-a1fd214fe02f None admin] RBAC: Authorizing `identity:get_user(user_id=e4316a12a39f42a08812c00e07f343a0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:08.253087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6cfbf-f4e7-4e0d-85d9-a1fd214fe02f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:08.254451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 97/191] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:08 2022] GET /identity/v3/users/e4316a12a39f42a08812c00e07f343a0 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:08.256467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6cfbf-f4e7-4e0d-85d9-a1fd214fe02f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:08.264358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64839f28-9ddb-4b8d-8622-69c3badee948 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:08.287566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.300049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6213df-a147-495b-a80f-615e45dd6dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:08.301177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6213df-a147-495b-a80f-615e45dd6dec None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.301460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6213df-a147-495b-a80f-615e45dd6dec None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.301732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6213df-a147-495b-a80f-615e45dd6dec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.302973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6213df-a147-495b-a80f-615e45dd6dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:08.304466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6213df-a147-495b-a80f-615e45dd6dec None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:08.306066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 95/192] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:08.307243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa77-2df6-46e0-bb27-2efa5e8a9595 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.307545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa77-2df6-46e0-bb27-2efa5e8a9595 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.307821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceaa77-2df6-46e0-bb27-2efa5e8a9595 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.309038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 96/193] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:08.309725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceaa77-2df6-46e0-bb27-2efa5e8a9595 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:08.348466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.356695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6cfbf-f4e7-4e0d-85d9-a1fd214fe02f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:08.390213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf457441-4da4-49f6-97a4-1f1305070026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:08.391089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf457441-4da4-49f6-97a4-1f1305070026 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.391412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf457441-4da4-49f6-97a4-1f1305070026 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.391691 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf457441-4da4-49f6-97a4-1f1305070026 None admin] PATH_INFO: `/v3/users/e4316a12a39f42a08812c00e07f343a0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.392874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf457441-4da4-49f6-97a4-1f1305070026 None admin] RBAC: Authorizing `identity:get_user(user_id=e4316a12a39f42a08812c00e07f343a0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:08.394319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf457441-4da4-49f6-97a4-1f1305070026 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:08.395971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 98/194] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:08 2022] GET /identity/v3/users/e4316a12a39f42a08812c00e07f343a0 => generated 283 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:08.396710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf457441-4da4-49f6-97a4-1f1305070026 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:08.412593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceaa77-2df6-46e0-bb27-2efa5e8a9595 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:08.445094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf308793-8e50-49d4-b5cf-ec56e4dfebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:08.445740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf308793-8e50-49d4-b5cf-ec56e4dfebaf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.446024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf308793-8e50-49d4-b5cf-ec56e4dfebaf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.446302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf308793-8e50-49d4-b5cf-ec56e4dfebaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.491744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf457441-4da4-49f6-97a4-1f1305070026 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:08.502443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.511556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf308793-8e50-49d4-b5cf-ec56e4dfebaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:08.513011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf308793-8e50-49d4-b5cf-ec56e4dfebaf None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:08.524179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a7b78-40f2-442f-b705-6b2b2e85ce29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:08.524865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7b78-40f2-442f-b705-6b2b2e85ce29 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.525147 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7b78-40f2-442f-b705-6b2b2e85ce29 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.525423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7b78-40f2-442f-b705-6b2b2e85ce29 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.526132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a7b78-40f2-442f-b705-6b2b2e85ce29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:08.527585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a7b78-40f2-442f-b705-6b2b2e85ce29 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:08.608563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 97/195] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:08.609817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0e78e-3786-4347-8571-c3e885eab00e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.610104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0e78e-3786-4347-8571-c3e885eab00e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.610378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0e78e-3786-4347-8571-c3e885eab00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.626178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4bd0e78e-3786-4347-8571-c3e885eab00e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:08.627379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4bd0e78e-3786-4347-8571-c3e885eab00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:08.650324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 99/196] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:08 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:08.651148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a7b78-40f2-442f-b705-6b2b2e85ce29 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:08.671236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.708207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 98/197] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:08.711470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0e78e-3786-4347-8571-c3e885eab00e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:08.742144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.774511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a7b78-40f2-442f-b705-6b2b2e85ce29 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:08.814356 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9817fd6-a5b8-430d-96d4-63303691467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:08.815295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9817fd6-a5b8-430d-96d4-63303691467c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.815295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9817fd6-a5b8-430d-96d4-63303691467c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.815605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9817fd6-a5b8-430d-96d4-63303691467c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.830124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0e78e-3786-4347-8571-c3e885eab00e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:08.870971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735e5af-8eeb-4ff8-9b2b-2f1d2bef8b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:08.870971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735e5af-8eeb-4ff8-9b2b-2f1d2bef8b72 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:08.870971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735e5af-8eeb-4ff8-9b2b-2f1d2bef8b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:08.870971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735e5af-8eeb-4ff8-9b2b-2f1d2bef8b72 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:08.873134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735e5af-8eeb-4ff8-9b2b-2f1d2bef8b72 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:08.873393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735e5af-8eeb-4ff8-9b2b-2f1d2bef8b72 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:08.875285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 99/198] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:08 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:08.880125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735e5af-8eeb-4ff8-9b2b-2f1d2bef8b72 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:08.897379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9817fd6-a5b8-430d-96d4-63303691467c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:08.899202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9817fd6-a5b8-430d-96d4-63303691467c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:08.925265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.986964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:08.999067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735e5af-8eeb-4ff8-9b2b-2f1d2bef8b72 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:09.036926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 100/199] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:09.037831 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9817fd6-a5b8-430d-96d4-63303691467c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:09.041189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7a51-7348-4991-9854-eaa35ae2eeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:09.042166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7a51-7348-4991-9854-eaa35ae2eeaf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:09.042544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7a51-7348-4991-9854-eaa35ae2eeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:09.042937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c7a51-7348-4991-9854-eaa35ae2eeaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:09.060699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c7a51-7348-4991-9854-eaa35ae2eeaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-eb2c7a51-7348-4991-9854-eaa35ae2eeaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:09.068721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:09.069807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:09.070806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:09.072656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 100/200] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:09.073642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7a51-7348-4991-9854-eaa35ae2eeaf None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:09.173221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9817fd6-a5b8-430d-96d4-63303691467c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:09.191237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:09.202936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c7a51-7348-4991-9854-eaa35ae2eeaf None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:09.210747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:09.212720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:09.212720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:09.212720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:09.213348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e, user.id=e4316a12a39f42a08812c00e07f343a0) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:09.213760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:09.215351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:09.223451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 101/201] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:09 2022] GET /identity/v3/role_assignments?user.id=e4316a12a39f42a08812c00e07f343a0&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=007af6bd1b734517b934b4cac45faf2c => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:09.224500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:09.247241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428337a-740d-4acb-951c-95461a53832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:09.248409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428337a-740d-4acb-951c-95461a53832f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:09.249388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428337a-740d-4acb-951c-95461a53832f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:09.249832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e428337a-740d-4acb-951c-95461a53832f None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:09.251696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428337a-740d-4acb-951c-95461a53832f None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:09.254044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e428337a-740d-4acb-951c-95461a53832f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:09.256565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 101/202] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:09 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:09.264573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428337a-740d-4acb-951c-95461a53832f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:09.315671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:09.413373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e428337a-740d-4acb-951c-95461a53832f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:09.426474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769564e4-685e-4e0a-9360-1f31f2299cfd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:09.458352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:09.459873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:09.460758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:09.461236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:09.462492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] RBAC: Adding query filter params (role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e, user.id=e4316a12a39f42a08812c00e07f343a0) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:09.463143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:09.465453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:09.476411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 102/203] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:09 2022] GET /identity/v3/role_assignments?user.id=e4316a12a39f42a08812c00e07f343a0&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:09.479970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:09.480883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:09.481343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:09.481791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:09.483382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:09.483889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:09.485504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:09.495675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 102/204] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:09.503390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:09.619605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:09.642498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c52715-fd48-4cbb-bb42-dd0f8761b422 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:09.684048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141ef53-8b81-44cf-90e0-cc752e044ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:09.684928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ef53-8b81-44cf-90e0-cc752e044ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:09.685741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ef53-8b81-44cf-90e0-cc752e044ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:09.686016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ef53-8b81-44cf-90e0-cc752e044ba2 None admin] PATH_INFO: `/v3/users/d07c08dd1c964eb89b7bc9d1ca53f68f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:09.691979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141ef53-8b81-44cf-90e0-cc752e044ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=d07c08dd1c964eb89b7bc9d1ca53f68f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:09.693436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141ef53-8b81-44cf-90e0-cc752e044ba2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:09.695412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 103/205] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:09 2022] GET /identity/v3/users/d07c08dd1c964eb89b7bc9d1ca53f68f => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:09.701965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:09.771894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:09.846040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e622082-2b15-4d38-83a0-c84f8a2d4b80 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:09.890739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14090c-ce45-40c1-991e-f73674bf5e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:09.891420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14090c-ce45-40c1-991e-f73674bf5e02 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:09.891707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14090c-ce45-40c1-991e-f73674bf5e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:09.891993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14090c-ce45-40c1-991e-f73674bf5e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:09.962614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14090c-ce45-40c1-991e-f73674bf5e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:09.964155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14090c-ce45-40c1-991e-f73674bf5e02 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:10.021015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:10.088291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 103/206] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:10.094462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141ef53-8b81-44cf-90e0-cc752e044ba2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:10.204871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141ef53-8b81-44cf-90e0-cc752e044ba2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:10.239874 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c0af4-e620-4fdc-a793-383ab077c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:10.240748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c0af4-e620-4fdc-a793-383ab077c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:10.241031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c0af4-e620-4fdc-a793-383ab077c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:10.241309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c0af4-e620-4fdc-a793-383ab077c1e8 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:10.242513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c0af4-e620-4fdc-a793-383ab077c1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:10.243976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c0af4-e620-4fdc-a793-383ab077c1e8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:10.245983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 104/207] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:10 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:10.252058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14090c-ce45-40c1-991e-f73674bf5e02 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:10.403075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14090c-ce45-40c1-991e-f73674bf5e02 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:10.439069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd43a3-d7b9-436a-93ad-d087329a1c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:10.440085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd43a3-d7b9-436a-93ad-d087329a1c8a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:10.440584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd43a3-d7b9-436a-93ad-d087329a1c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:10.441055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd43a3-d7b9-436a-93ad-d087329a1c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:10.442187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd43a3-d7b9-436a-93ad-d087329a1c8a None admin] RBAC: Adding query filter params (role.id=3582abcd4fa246d4add3f71e7474643f, scope.project.id=6ca74ec2a86e48a28a003d344862f755, user.id=d07c08dd1c964eb89b7bc9d1ca53f68f) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:10.442706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd43a3-d7b9-436a-93ad-d087329a1c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:10.444375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd43a3-d7b9-436a-93ad-d087329a1c8a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:10.450960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 104/208] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:10 2022] GET /identity/v3/role_assignments?user.id=d07c08dd1c964eb89b7bc9d1ca53f68f&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=3582abcd4fa246d4add3f71e7474643f => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:11.035610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36063a23-d446-4f14-8ab7-b8abab13f357 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.036004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36063a23-d446-4f14-8ab7-b8abab13f357 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.036201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36063a23-d446-4f14-8ab7-b8abab13f357 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.037685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 105/209] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:11.043683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485913c-850e-4c42-8aa0-326f85e91fcf None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.044604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485913c-850e-4c42-8aa0-326f85e91fcf None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.045085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485913c-850e-4c42-8aa0-326f85e91fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.057966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:11.071218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7485913c-850e-4c42-8aa0-326f85e91fcf None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:11.075282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7485913c-850e-4c42-8aa0-326f85e91fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:11.162277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 105/210] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:11.185613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36063a23-d446-4f14-8ab7-b8abab13f357 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:11.206831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:11.299770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36063a23-d446-4f14-8ab7-b8abab13f357 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:11.328751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488341-f019-4ea1-af1c-ae15d4b358b5 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.329145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488341-f019-4ea1-af1c-ae15d4b358b5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.329432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25488341-f019-4ea1-af1c-ae15d4b358b5 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.330926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 106/211] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:11.336019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e2c5d-74ed-40f5-bb08-adbdbdac9735 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.336306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e2c5d-74ed-40f5-bb08-adbdbdac9735 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.336578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e2c5d-74ed-40f5-bb08-adbdbdac9735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.337656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d11d5-9a91-4d3b-913f-34fd0a5ac3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:11.338514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d11d5-9a91-4d3b-913f-34fd0a5ac3da None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.338798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d11d5-9a91-4d3b-913f-34fd0a5ac3da None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.339078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d11d5-9a91-4d3b-913f-34fd0a5ac3da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.349996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d11d5-9a91-4d3b-913f-34fd0a5ac3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:11.354111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fa4e2c5d-74ed-40f5-bb08-adbdbdac9735 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:11.355420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fa4e2c5d-74ed-40f5-bb08-adbdbdac9735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1c4d11d5-9a91-4d3b-913f-34fd0a5ac3da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:11.358380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:11.359460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:11.360566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 106/212] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:11.365546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d11d5-9a91-4d3b-913f-34fd0a5ac3da None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:11.445927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 107/213] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:11.447039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:11.471239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e2c5d-74ed-40f5-bb08-adbdbdac9735 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:11.472464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d11d5-9a91-4d3b-913f-34fd0a5ac3da None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:11.514847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:11.515969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.516254 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.516531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.517281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:11.517589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:11.519367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:11.530240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 107/214] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:11.535049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:11.547893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:11.585893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e2c5d-74ed-40f5-bb08-adbdbdac9735 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:11.624170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e7131-2d77-422d-b2b5-ebd897ba919a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:11.625494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e7131-2d77-422d-b2b5-ebd897ba919a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.625972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e7131-2d77-422d-b2b5-ebd897ba919a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.626455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e7131-2d77-422d-b2b5-ebd897ba919a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.627871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e7131-2d77-422d-b2b5-ebd897ba919a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:11.629611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e7131-2d77-422d-b2b5-ebd897ba919a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:11.631535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 108/215] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:11.636862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e7131-2d77-422d-b2b5-ebd897ba919a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:11.669873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9083e-ec30-4483-8bcf-b443f3b5b97f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:11.704873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e56113-5529-4248-a66c-0eee78ed6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:11.705985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e56113-5529-4248-a66c-0eee78ed6279 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.706450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e56113-5529-4248-a66c-0eee78ed6279 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.706871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e56113-5529-4248-a66c-0eee78ed6279 None admin] PATH_INFO: `/v3/users/93ad569cd8264f20919c5d38c60ae507` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.719348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:11.723525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e56113-5529-4248-a66c-0eee78ed6279 None admin] RBAC: Authorizing `identity:get_user(user_id=93ad569cd8264f20919c5d38c60ae507)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:11.724984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e56113-5529-4248-a66c-0eee78ed6279 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:11.726696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 108/216] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:11 2022] GET /identity/v3/users/93ad569cd8264f20919c5d38c60ae507 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:11.726696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a2816-b034-4560-85f0-06c0d2d92f76 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.726696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a2816-b034-4560-85f0-06c0d2d92f76 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.726696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a2816-b034-4560-85f0-06c0d2d92f76 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.726696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 109/217] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:11.734965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a2816-b034-4560-85f0-06c0d2d92f76 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:11.749454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e7131-2d77-422d-b2b5-ebd897ba919a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:11.785240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae236f-592e-4f86-933d-9a1095bc668d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:11.785909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae236f-592e-4f86-933d-9a1095bc668d None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.786189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae236f-592e-4f86-933d-9a1095bc668d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.786463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae236f-592e-4f86-933d-9a1095bc668d None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.787210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae236f-592e-4f86-933d-9a1095bc668d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:11.788669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae236f-592e-4f86-933d-9a1095bc668d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:11.795806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:11.840510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a2816-b034-4560-85f0-06c0d2d92f76 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:11.874878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773fca5-abf6-445f-b9b0-458143068db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:11.875545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773fca5-abf6-445f-b9b0-458143068db7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.875825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773fca5-abf6-445f-b9b0-458143068db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.876097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773fca5-abf6-445f-b9b0-458143068db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.907621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 109/218] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:11 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:05:11.909036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60ce24-b643-41a7-b34f-71e935c7b798 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:11.909327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60ce24-b643-41a7-b34f-71e935c7b798 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:11.909608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60ce24-b643-41a7-b34f-71e935c7b798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:11.928475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3b60ce24-b643-41a7-b34f-71e935c7b798 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:11.929710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3b60ce24-b643-41a7-b34f-71e935c7b798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:11.953283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:11.957373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773fca5-abf6-445f-b9b0-458143068db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:11.958872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773fca5-abf6-445f-b9b0-458143068db7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:12.007365 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 110/219] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:12.008723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260d0b8-1356-44d9-8d18-cdb8dd868602 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:12.009016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260d0b8-1356-44d9-8d18-cdb8dd868602 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:12.009296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260d0b8-1356-44d9-8d18-cdb8dd868602 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:12.010607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 111/220] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:12.017246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c4b0d-fae1-4250-8aa9-81e51903067b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:12.017541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c4b0d-fae1-4250-8aa9-81e51903067b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:12.017810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c4b0d-fae1-4250-8aa9-81e51903067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:12.038009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-644c4b0d-fae1-4250-8aa9-81e51903067b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:12.039534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-644c4b0d-fae1-4250-8aa9-81e51903067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:12.100525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 110/221] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:12.101772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773fca5-abf6-445f-b9b0-458143068db7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:12.123338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 112/222] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:12.124064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c4b0d-fae1-4250-8aa9-81e51903067b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:12.180754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:12.210258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:12.215825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773fca5-abf6-445f-b9b0-458143068db7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:12.248164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c4b0d-fae1-4250-8aa9-81e51903067b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:12.258411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefab93c-1cfa-47a4-99b1-75e65f8acb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:12.259368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab93c-1cfa-47a4-99b1-75e65f8acb29 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:12.259653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab93c-1cfa-47a4-99b1-75e65f8acb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:12.259928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab93c-1cfa-47a4-99b1-75e65f8acb29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:12.279282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefab93c-1cfa-47a4-99b1-75e65f8acb29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:12.284603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a134-2e68-42e5-ac24-3b97c025c051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:12.285466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a134-2e68-42e5-ac24-3b97c025c051 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:12.285745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a134-2e68-42e5-ac24-3b97c025c051 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:12.286018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a134-2e68-42e5-ac24-3b97c025c051 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:12.287211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2a134-2e68-42e5-ac24-3b97c025c051 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:12.288707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2a134-2e68-42e5-ac24-3b97c025c051 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-aefab93c-1cfa-47a4-99b1-75e65f8acb29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:12.289822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:12.290680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:12.291556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:12.292171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 113/223] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:12 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:12.292171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a134-2e68-42e5-ac24-3b97c025c051 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:12.300872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 111/224] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:12.302013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefab93c-1cfa-47a4-99b1-75e65f8acb29 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:12.412316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefab93c-1cfa-47a4-99b1-75e65f8acb29 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:12.427048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a134-2e68-42e5-ac24-3b97c025c051 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:12.445323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:12.458611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:12.465088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:12.465088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:12.465088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:12.465088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] RBAC: Adding query filter params (role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e, user.id=93ad569cd8264f20919c5d38c60ae507) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:12.465088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:12.465088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:12.465602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46324778-e990-4360-a471-d71d8df6b9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:12.465602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46324778-e990-4360-a471-d71d8df6b9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:12.466136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46324778-e990-4360-a471-d71d8df6b9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:12.466692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46324778-e990-4360-a471-d71d8df6b9d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:12.475284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 112/225] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:12 2022] GET /identity/v3/role_assignments?user.id=93ad569cd8264f20919c5d38c60ae507&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:12.475284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:12.492085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46324778-e990-4360-a471-d71d8df6b9d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:12.500483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-46324778-e990-4360-a471-d71d8df6b9d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:12.506109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:12.507012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:12.507860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:12.515286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 114/226] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:05:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:12.516988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46324778-e990-4360-a471-d71d8df6b9d1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:12.605992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852424f2-5cf1-4d22-ac0e-fdaa298eb70a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:12.647288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:12.647972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:12.648260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:12.648540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:12.649298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:12.649610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:12.651017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:12.660713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 113/227] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:12.666898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:12.728544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46324778-e990-4360-a471-d71d8df6b9d1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:12.796230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:12.796914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:12.797257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:12.797957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:12.799019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:12.799372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:12.801258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:12.802068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:12.815276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 115/228] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:12.817408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:12.834132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167ef82-522e-4962-808e-cf84c3fe311c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:12.963687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:13.019965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360802-05a3-491d-bb0f-786e720e7d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:13.020855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360802-05a3-491d-bb0f-786e720e7d02 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:13.021148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360802-05a3-491d-bb0f-786e720e7d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:13.021426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81360802-05a3-491d-bb0f-786e720e7d02 None admin] PATH_INFO: `/v3/users/e4316a12a39f42a08812c00e07f343a0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:13.023764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81360802-05a3-491d-bb0f-786e720e7d02 None admin] RBAC: Authorizing `identity:get_user(user_id=e4316a12a39f42a08812c00e07f343a0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:13.027650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81360802-05a3-491d-bb0f-786e720e7d02 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:13.030066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 114/229] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:12 2022] GET /identity/v3/users/e4316a12a39f42a08812c00e07f343a0 => generated 283 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:13.035355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360802-05a3-491d-bb0f-786e720e7d02 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:13.067636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a98f39-19b7-4225-8edd-40656c5d4fb0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:13.114212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa02d5-2e19-4b11-87bd-a5961afa35d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:13.115497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa02d5-2e19-4b11-87bd-a5961afa35d1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:13.117230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa02d5-2e19-4b11-87bd-a5961afa35d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:13.117857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa02d5-2e19-4b11-87bd-a5961afa35d1 None admin] PATH_INFO: `/v3/users/e4316a12a39f42a08812c00e07f343a0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:13.121797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa02d5-2e19-4b11-87bd-a5961afa35d1 None admin] RBAC: Authorizing `identity:get_user(user_id=e4316a12a39f42a08812c00e07f343a0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:13.128839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa02d5-2e19-4b11-87bd-a5961afa35d1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:13.131860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 116/230] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:12 2022] GET /identity/v3/users/e4316a12a39f42a08812c00e07f343a0 => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:13.139927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa02d5-2e19-4b11-87bd-a5961afa35d1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:13.208034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81360802-05a3-491d-bb0f-786e720e7d02 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:13.268784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fe51b-ec78-404f-8d90-a66b894fc6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:13.270597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fe51b-ec78-404f-8d90-a66b894fc6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:13.271987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fe51b-ec78-404f-8d90-a66b894fc6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:13.272599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fe51b-ec78-404f-8d90-a66b894fc6e7 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:13.275099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fe51b-ec78-404f-8d90-a66b894fc6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:13.278051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260fe51b-ec78-404f-8d90-a66b894fc6e7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:13.281995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 115/231] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:13 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:13.286208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fe51b-ec78-404f-8d90-a66b894fc6e7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:13.327937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:13.341876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa02d5-2e19-4b11-87bd-a5961afa35d1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:13.365479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:13.424590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd6596-fa03-4c7e-b09a-bb63253221f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:13.426128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6596-fa03-4c7e-b09a-bb63253221f7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:13.426612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6596-fa03-4c7e-b09a-bb63253221f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:13.427087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6596-fa03-4c7e-b09a-bb63253221f7 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:13.428514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd6596-fa03-4c7e-b09a-bb63253221f7 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:13.430164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd6596-fa03-4c7e-b09a-bb63253221f7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:13.430764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fe51b-ec78-404f-8d90-a66b894fc6e7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:13.432335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 117/232] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:13 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:13.437993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd6596-fa03-4c7e-b09a-bb63253221f7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:13.470252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e52d9-a32a-4d19-a925-e74b94e43c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:13.472490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e52d9-a32a-4d19-a925-e74b94e43c32 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:13.474429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e52d9-a32a-4d19-a925-e74b94e43c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:13.475776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e52d9-a32a-4d19-a925-e74b94e43c32 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/users/e4316a12a39f42a08812c00e07f343a0/roles/9eaabfe45189403f8cee9512fefbba65` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:13.481956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e52d9-a32a-4d19-a925-e74b94e43c32 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf653a9fe09a4a3e86a6e3e8c134711e, user_id=e4316a12a39f42a08812c00e07f343a0, role_id=9eaabfe45189403f8cee9512fefbba65)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:13.483808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e52d9-a32a-4d19-a925-e74b94e43c32 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:13.570875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd6596-fa03-4c7e-b09a-bb63253221f7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:13.574516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 116/233] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:13 2022] PUT /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/users/e4316a12a39f42a08812c00e07f343a0/roles/9eaabfe45189403f8cee9512fefbba65 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:13.614909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82403d-6125-4053-ab7c-713af0dac713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:13.616305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82403d-6125-4053-ab7c-713af0dac713 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:13.616786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82403d-6125-4053-ab7c-713af0dac713 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:13.617321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82403d-6125-4053-ab7c-713af0dac713 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/users/e4316a12a39f42a08812c00e07f343a0/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:13.619633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82403d-6125-4053-ab7c-713af0dac713 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf653a9fe09a4a3e86a6e3e8c134711e, user_id=e4316a12a39f42a08812c00e07f343a0, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:13.621570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82403d-6125-4053-ab7c-713af0dac713 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:13.731121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 118/234] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:13 2022] PUT /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/users/e4316a12a39f42a08812c00e07f343a0/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:14.292359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386507-a3ee-4430-9a86-ddb13ca537b1 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:14.293417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386507-a3ee-4430-9a86-ddb13ca537b1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:14.293899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386507-a3ee-4430-9a86-ddb13ca537b1 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:14.296162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 117/235] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:14.296365 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527b75-b8f5-4e1d-afaa-e48f3f2f9437 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:14.296756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527b75-b8f5-4e1d-afaa-e48f3f2f9437 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:14.296909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527b75-b8f5-4e1d-afaa-e48f3f2f9437 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:14.298291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 119/236] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:14.302332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3445da-b1cc-412c-bc57-5c4fcb2f2265 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:14.302733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3445da-b1cc-412c-bc57-5c4fcb2f2265 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:14.303162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3445da-b1cc-412c-bc57-5c4fcb2f2265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:14.305837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee59ec-4a7f-4178-95a8-719a98652be1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:14.306582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee59ec-4a7f-4178-95a8-719a98652be1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:14.307506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee59ec-4a7f-4178-95a8-719a98652be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:14.332928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9f3445da-b1cc-412c-bc57-5c4fcb2f2265 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:14.335986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9f3445da-b1cc-412c-bc57-5c4fcb2f2265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:14.352354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0dee59ec-4a7f-4178-95a8-719a98652be1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:14.355553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0dee59ec-4a7f-4178-95a8-719a98652be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:14.417099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:14.457111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 118/237] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:14.471075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 120/238] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:14.483385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3445da-b1cc-412c-bc57-5c4fcb2f2265 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:14.495630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:14.507743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee59ec-4a7f-4178-95a8-719a98652be1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:14.585714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3445da-b1cc-412c-bc57-5c4fcb2f2265 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:14.623155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2043d-923e-44f6-b131-9407c96e5bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:14.624717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2043d-923e-44f6-b131-9407c96e5bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:14.625478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2043d-923e-44f6-b131-9407c96e5bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:14.625957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2043d-923e-44f6-b131-9407c96e5bb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:14.627365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2043d-923e-44f6-b131-9407c96e5bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:14.629368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2043d-923e-44f6-b131-9407c96e5bb2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:14.636074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 119/239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:14.637740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2043d-923e-44f6-b131-9407c96e5bb2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:14.727645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:14.739838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee59ec-4a7f-4178-95a8-719a98652be1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:14.749296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2043d-923e-44f6-b131-9407c96e5bb2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:14.787900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8467df5-d51f-4224-83ca-ebc95a72cfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:14.788940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8467df5-d51f-4224-83ca-ebc95a72cfb9 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:14.789421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8467df5-d51f-4224-83ca-ebc95a72cfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:14.789893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8467df5-d51f-4224-83ca-ebc95a72cfb9 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:14.791292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8467df5-d51f-4224-83ca-ebc95a72cfb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:14.793113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8467df5-d51f-4224-83ca-ebc95a72cfb9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:14.820775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580d581-c108-46cf-9f09-9f79abf28487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:14.821639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580d581-c108-46cf-9f09-9f79abf28487 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:14.821925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580d581-c108-46cf-9f09-9f79abf28487 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:14.822340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580d581-c108-46cf-9f09-9f79abf28487 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:14.835425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580d581-c108-46cf-9f09-9f79abf28487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a580d581-c108-46cf-9f09-9f79abf28487 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:14.845134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:14.846618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:14.847514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 121/240] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:14.851217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580d581-c108-46cf-9f09-9f79abf28487 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:14.884545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 120/241] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:14 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:05:14.891281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:14.971973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580d581-c108-46cf-9f09-9f79abf28487 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:14.986196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41b1b1-878d-4f61-99f6-29e67a2904f5 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:14.986487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41b1b1-878d-4f61-99f6-29e67a2904f5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:14.986763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41b1b1-878d-4f61-99f6-29e67a2904f5 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:14.988427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 121/242] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:15.000828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d332ada-b12c-42ee-9b1c-1dabf6de58f5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.001134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d332ada-b12c-42ee-9b1c-1dabf6de58f5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.001391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d332ada-b12c-42ee-9b1c-1dabf6de58f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.009785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:15.010489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.010731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.011006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.011803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:15.012110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:15.013595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:15.024755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 122/243] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:15.031543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:15.048391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3d332ada-b12c-42ee-9b1c-1dabf6de58f5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:15.049639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3d332ada-b12c-42ee-9b1c-1dabf6de58f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:15.154974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:15.172905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d19367-56df-4275-8831-747fcdc3ed10 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:15.176515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 122/244] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:15.199348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d332ada-b12c-42ee-9b1c-1dabf6de58f5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:15.211664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadda63-0101-4afb-b6d6-e169c991b3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:15.212553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadda63-0101-4afb-b6d6-e169c991b3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.212839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadda63-0101-4afb-b6d6-e169c991b3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.213108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadda63-0101-4afb-b6d6-e169c991b3f2 None admin] PATH_INFO: `/v3/users/594b774f0bf048e9adde846098b8d1fc` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.230155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadda63-0101-4afb-b6d6-e169c991b3f2 None admin] RBAC: Authorizing `identity:get_user(user_id=594b774f0bf048e9adde846098b8d1fc)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:15.233028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cadda63-0101-4afb-b6d6-e169c991b3f2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:15.235270 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 123/245] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:15 2022] GET /identity/v3/users/594b774f0bf048e9adde846098b8d1fc => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:15.240605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadda63-0101-4afb-b6d6-e169c991b3f2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:15.273339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:15.329694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d332ada-b12c-42ee-9b1c-1dabf6de58f5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:15.374547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadda63-0101-4afb-b6d6-e169c991b3f2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:15.378431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d84de-2c86-4d6e-81e0-c79aa9cdff63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:15.380156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d84de-2c86-4d6e-81e0-c79aa9cdff63 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.380642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d84de-2c86-4d6e-81e0-c79aa9cdff63 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.381135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d84de-2c86-4d6e-81e0-c79aa9cdff63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.396062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25d84de-2c86-4d6e-81e0-c79aa9cdff63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e25d84de-2c86-4d6e-81e0-c79aa9cdff63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:15.404708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:15.405638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:15.406480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:15.409183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 123/246] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:15.415362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d84de-2c86-4d6e-81e0-c79aa9cdff63 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:15.422832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a43eac-4389-4bf0-b1ea-19fe66aae4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:15.423904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43eac-4389-4bf0-b1ea-19fe66aae4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.423904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43eac-4389-4bf0-b1ea-19fe66aae4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.424155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43eac-4389-4bf0-b1ea-19fe66aae4a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.447930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:15.529676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a43eac-4389-4bf0-b1ea-19fe66aae4a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:15.531560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a43eac-4389-4bf0-b1ea-19fe66aae4a2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:15.558058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d84de-2c86-4d6e-81e0-c79aa9cdff63 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:15.590709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:15.610945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:15.612259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.612745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.613461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.615033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:15.615655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:15.617875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:15.632254 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 124/247] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:15.638847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:15.666771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 124/248] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:05:15.672838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a43eac-4389-4bf0-b1ea-19fe66aae4a2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:15.757220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:15.810672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff521ba-aeaf-4f70-b719-bd1ebb5c8d1c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:15.858080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a43eac-4389-4bf0-b1ea-19fe66aae4a2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:15.891279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004918c8-88ac-4513-8483-55da2f5c9b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:15.891279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004918c8-88ac-4513-8483-55da2f5c9b24 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.891279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004918c8-88ac-4513-8483-55da2f5c9b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.891279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004918c8-88ac-4513-8483-55da2f5c9b24 None admin] PATH_INFO: `/v3/users/93ad569cd8264f20919c5d38c60ae507` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.893119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004918c8-88ac-4513-8483-55da2f5c9b24 None admin] RBAC: Authorizing `identity:get_user(user_id=93ad569cd8264f20919c5d38c60ae507)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:15.894759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004918c8-88ac-4513-8483-55da2f5c9b24 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:15.897561 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 125/249] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:15 2022] GET /identity/v3/users/93ad569cd8264f20919c5d38c60ae507 => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:15.898427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848cd622-9700-4d33-9adb-4e2814779769 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.898711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848cd622-9700-4d33-9adb-4e2814779769 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.898988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848cd622-9700-4d33-9adb-4e2814779769 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.900258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 126/250] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:15.909924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848cd622-9700-4d33-9adb-4e2814779769 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:15.920776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0607368-739d-4cec-98e3-7d14b21c804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:15.921575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607368-739d-4cec-98e3-7d14b21c804a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.921856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607368-739d-4cec-98e3-7d14b21c804a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.922132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0607368-739d-4cec-98e3-7d14b21c804a None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.927797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0607368-739d-4cec-98e3-7d14b21c804a None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:15.929269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0607368-739d-4cec-98e3-7d14b21c804a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:15.931016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 125/251] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:15 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:15.932232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691173b7-8809-4abf-a8d4-32fb4457dfef None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:15.932517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691173b7-8809-4abf-a8d4-32fb4457dfef None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:15.932797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691173b7-8809-4abf-a8d4-32fb4457dfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:15.959094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-691173b7-8809-4abf-a8d4-32fb4457dfef None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:15.960507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-691173b7-8809-4abf-a8d4-32fb4457dfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:16.043032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 126/252] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:16.043610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:16.046358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691173b7-8809-4abf-a8d4-32fb4457dfef None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:16.130000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848cd622-9700-4d33-9adb-4e2814779769 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:16.142031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:16.200197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523ee22-be01-46a7-b23c-d21d47e05284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:16.201169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523ee22-be01-46a7-b23c-d21d47e05284 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:16.201453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523ee22-be01-46a7-b23c-d21d47e05284 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:16.201728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523ee22-be01-46a7-b23c-d21d47e05284 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:16.205605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523ee22-be01-46a7-b23c-d21d47e05284 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:16.207514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523ee22-be01-46a7-b23c-d21d47e05284 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:16.209922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 127/253] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:15 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:16.210840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523ee22-be01-46a7-b23c-d21d47e05284 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:16.221287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691173b7-8809-4abf-a8d4-32fb4457dfef None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:16.260636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:16.261395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:16.261751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:16.262103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:16.263081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] RBAC: Adding query filter params (role.id=3582abcd4fa246d4add3f71e7474643f, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e, user.id=594b774f0bf048e9adde846098b8d1fc) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:16.263484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:16.265585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:16.272647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 127/254] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:16 2022] GET /identity/v3/role_assignments?user.id=594b774f0bf048e9adde846098b8d1fc&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=3582abcd4fa246d4add3f71e7474643f => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:16.273895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:16.391845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46384cc-9ec4-4a76-8be8-09591bb3b9ce None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:16.415750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:16.431068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bba73e-2b96-448d-a0ae-6596a5239693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:16.432886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bba73e-2b96-448d-a0ae-6596a5239693 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:16.433247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bba73e-2b96-448d-a0ae-6596a5239693 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:16.433597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bba73e-2b96-448d-a0ae-6596a5239693 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/users/93ad569cd8264f20919c5d38c60ae507/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:16.436740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bba73e-2b96-448d-a0ae-6596a5239693 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf653a9fe09a4a3e86a6e3e8c134711e, user_id=93ad569cd8264f20919c5d38c60ae507, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:16.438575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bba73e-2b96-448d-a0ae-6596a5239693 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:16.487312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523ee22-be01-46a7-b23c-d21d47e05284 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:16.513293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:16.515435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 128/255] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:16 2022] PUT /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/users/93ad569cd8264f20919c5d38c60ae507/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:16.531324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87c413-26c5-4c91-930c-d497250480c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:16.532227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87c413-26c5-4c91-930c-d497250480c4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:16.532510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87c413-26c5-4c91-930c-d497250480c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:16.532833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87c413-26c5-4c91-930c-d497250480c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:16.548321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87c413-26c5-4c91-930c-d497250480c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8f87c413-26c5-4c91-930c-d497250480c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:16.557053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:16.558037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:16.558863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:16.563661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 128/256] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:16.564569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bba73e-2b96-448d-a0ae-6596a5239693 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:16.719413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bba73e-2b96-448d-a0ae-6596a5239693 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:16.763293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a62070-4aaf-4743-9dbe-8e464f19ea4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:16.763903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62070-4aaf-4743-9dbe-8e464f19ea4b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:16.764185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62070-4aaf-4743-9dbe-8e464f19ea4b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:16.764460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62070-4aaf-4743-9dbe-8e464f19ea4b None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:16.765234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a62070-4aaf-4743-9dbe-8e464f19ea4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:16.765543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a62070-4aaf-4743-9dbe-8e464f19ea4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:16.766971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a62070-4aaf-4743-9dbe-8e464f19ea4b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:16.779827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 129/257] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:16.785062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87c413-26c5-4c91-930c-d497250480c4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:16.815202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3821f-74b2-4c49-882b-3b9110f8c992 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:16.815522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3821f-74b2-4c49-882b-3b9110f8c992 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:16.815792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3821f-74b2-4c49-882b-3b9110f8c992 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:16.817665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 130/258] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:16.823342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed796e31-68d8-46ec-ac7b-b716e66a3320 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:16.823634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed796e31-68d8-46ec-ac7b-b716e66a3320 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:16.823915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed796e31-68d8-46ec-ac7b-b716e66a3320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:16.843245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ed796e31-68d8-46ec-ac7b-b716e66a3320 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:16.844546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ed796e31-68d8-46ec-ac7b-b716e66a3320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:16.874963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:16.935376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 131/259] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:16.954728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87c413-26c5-4c91-930c-d497250480c4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:16.959279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed796e31-68d8-46ec-ac7b-b716e66a3320 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:17.021804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7bd84-6a91-4457-a30f-c6f92906b8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:17.022637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7bd84-6a91-4457-a30f-c6f92906b8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:17.022923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7bd84-6a91-4457-a30f-c6f92906b8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:17.023200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7bd84-6a91-4457-a30f-c6f92906b8a1 None admin] PATH_INFO: `/v3/users/e4316a12a39f42a08812c00e07f343a0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:17.024810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7bd84-6a91-4457-a30f-c6f92906b8a1 None admin] RBAC: Authorizing `identity:get_user(user_id=e4316a12a39f42a08812c00e07f343a0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:17.029098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7bd84-6a91-4457-a30f-c6f92906b8a1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:17.031622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 129/260] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:16 2022] GET /identity/v3/users/e4316a12a39f42a08812c00e07f343a0 => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:17.036853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7bd84-6a91-4457-a30f-c6f92906b8a1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:17.389396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:17.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed796e31-68d8-46ec-ac7b-b716e66a3320 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:17.389695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7bd84-6a91-4457-a30f-c6f92906b8a1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:17.389772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f15e0-cdfa-4ddb-bac5-804cb8e6f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:17.389772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f15e0-cdfa-4ddb-bac5-804cb8e6f3af None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:17.389772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f15e0-cdfa-4ddb-bac5-804cb8e6f3af None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:17.389772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f15e0-cdfa-4ddb-bac5-804cb8e6f3af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:17.390046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f15e0-cdfa-4ddb-bac5-804cb8e6f3af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:17.390118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb5540-93be-4145-95d7-8c069032fa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:17.390118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb5540-93be-4145-95d7-8c069032fa06 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:17.390118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb5540-93be-4145-95d7-8c069032fa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:17.390118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb5540-93be-4145-95d7-8c069032fa06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-620f15e0-cdfa-4ddb-bac5-804cb8e6f3af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:17.390381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:17.391221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 132/261] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:05:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:17.392549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f15e0-cdfa-4ddb-bac5-804cb8e6f3af None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:17.393296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:17.409529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb5540-93be-4145-95d7-8c069032fa06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:17.411046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb5540-93be-4145-95d7-8c069032fa06 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:17.524297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:17.546244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f15e0-cdfa-4ddb-bac5-804cb8e6f3af None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:17.582102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:17.582773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:17.583059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:17.583391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:17.584155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:17.584469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:17.585875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:17.594892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 133/262] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:17.609239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:17.617040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 130/263] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:17.623155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb5540-93be-4145-95d7-8c069032fa06 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:17.748587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:17.798036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0413c5f-5b06-4234-95de-ea3da4840660 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:17.848032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb5540-93be-4145-95d7-8c069032fa06 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:17.897701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4afd4d-5e98-4092-a0d0-47ef7fcaccc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:17.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4afd4d-5e98-4092-a0d0-47ef7fcaccc5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:17.898850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4afd4d-5e98-4092-a0d0-47ef7fcaccc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:17.899128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4afd4d-5e98-4092-a0d0-47ef7fcaccc5 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:17.900667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4afd4d-5e98-4092-a0d0-47ef7fcaccc5 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:17.902135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4afd4d-5e98-4092-a0d0-47ef7fcaccc5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:17.905785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4ac28-c3d0-4206-a8fb-eb4a149adc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:17.906674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4ac28-c3d0-4206-a8fb-eb4a149adc85 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:17.906954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4ac28-c3d0-4206-a8fb-eb4a149adc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:17.907227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4ac28-c3d0-4206-a8fb-eb4a149adc85 None admin] PATH_INFO: `/v3/users/e4316a12a39f42a08812c00e07f343a0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:17.908532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4ac28-c3d0-4206-a8fb-eb4a149adc85 None admin] RBAC: Authorizing `identity:get_user(user_id=e4316a12a39f42a08812c00e07f343a0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:17.910115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4ac28-c3d0-4206-a8fb-eb4a149adc85 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:17.911780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 134/264] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:17 2022] GET /identity/v3/users/e4316a12a39f42a08812c00e07f343a0 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:17.915583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 131/265] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:17 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:17.916312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4ac28-c3d0-4206-a8fb-eb4a149adc85 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:17.921773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4afd4d-5e98-4092-a0d0-47ef7fcaccc5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:17.944672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:18.032176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4afd4d-5e98-4092-a0d0-47ef7fcaccc5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:18.078974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b5b67-8835-4e12-9031-081643697373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:18.079661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b5b67-8835-4e12-9031-081643697373 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.079944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b5b67-8835-4e12-9031-081643697373 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.080221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b5b67-8835-4e12-9031-081643697373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.146001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4ac28-c3d0-4206-a8fb-eb4a149adc85 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:18.206521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b5b67-8835-4e12-9031-081643697373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:18.208018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b5b67-8835-4e12-9031-081643697373 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:18.209391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1278fa-93d2-453e-87c0-b1d8b22da345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:18.210296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1278fa-93d2-453e-87c0-b1d8b22da345 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.210773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1278fa-93d2-453e-87c0-b1d8b22da345 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.211268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1278fa-93d2-453e-87c0-b1d8b22da345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.212429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1278fa-93d2-453e-87c0-b1d8b22da345 None admin] RBAC: Adding query filter params (role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e, user.id=e4316a12a39f42a08812c00e07f343a0) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:18.212951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1278fa-93d2-453e-87c0-b1d8b22da345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:18.214590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1278fa-93d2-453e-87c0-b1d8b22da345 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:18.227285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 135/266] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:17 2022] GET /identity/v3/role_assignments?user.id=e4316a12a39f42a08812c00e07f343a0&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:18.227497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:18.319019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0e66b-02fa-4007-98b2-fee27ed63d59 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.319660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0e66b-02fa-4007-98b2-fee27ed63d59 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.320248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0e66b-02fa-4007-98b2-fee27ed63d59 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.322003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 136/267] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:18.331058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ac42-8cb1-4124-a509-b9d654734324 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.331788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ac42-8cb1-4124-a509-b9d654734324 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.332263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ac42-8cb1-4124-a509-b9d654734324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.338299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 132/268] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 11:05:18.338299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b5b67-8835-4e12-9031-081643697373 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:18.371136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0175ac42-8cb1-4124-a509-b9d654734324 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:18.373397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0175ac42-8cb1-4124-a509-b9d654734324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:18.391403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:18.526466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b5b67-8835-4e12-9031-081643697373 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:18.553448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 137/269] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:18.557852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:18.584978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123803-b9f2-4185-ab98-c88ea22f4f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:18.586075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123803-b9f2-4185-ab98-c88ea22f4f1b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.586521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123803-b9f2-4185-ab98-c88ea22f4f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.586946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123803-b9f2-4185-ab98-c88ea22f4f1b None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.588371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49123803-b9f2-4185-ab98-c88ea22f4f1b None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:18.590019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49123803-b9f2-4185-ab98-c88ea22f4f1b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:18.600716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175ac42-8cb1-4124-a509-b9d654734324 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:18.616729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 133/270] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:18 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:18.617969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123803-b9f2-4185-ab98-c88ea22f4f1b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:18.736524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123803-b9f2-4185-ab98-c88ea22f4f1b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:18.796187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15d4d2-dcd6-4c9f-9543-0556b954ab1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:18.797423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15d4d2-dcd6-4c9f-9543-0556b954ab1f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.798028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15d4d2-dcd6-4c9f-9543-0556b954ab1f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.798519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15d4d2-dcd6-4c9f-9543-0556b954ab1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.815218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175ac42-8cb1-4124-a509-b9d654734324 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:18.824498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15d4d2-dcd6-4c9f-9543-0556b954ab1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cb15d4d2-dcd6-4c9f-9543-0556b954ab1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:18.845884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:18.846863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:18.848449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 134/271] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:18.852306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceda4db-7e39-42a2-8f56-a2785a0c9f03 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.853583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceda4db-7e39-42a2-8f56-a2785a0c9f03 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.854426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceda4db-7e39-42a2-8f56-a2785a0c9f03 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.858720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 135/272] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:18 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:18.859856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceda4db-7e39-42a2-8f56-a2785a0c9f03 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:18.908865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b538abe-7ede-458b-ac41-d81bc9937df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:18.909539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b538abe-7ede-458b-ac41-d81bc9937df3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.909829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b538abe-7ede-458b-ac41-d81bc9937df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.910111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b538abe-7ede-458b-ac41-d81bc9937df3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.910966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b538abe-7ede-458b-ac41-d81bc9937df3 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e, user.id=e4316a12a39f42a08812c00e07f343a0) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:18.911298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b538abe-7ede-458b-ac41-d81bc9937df3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:18.913753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b538abe-7ede-458b-ac41-d81bc9937df3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:18.930423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 138/273] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:18 2022] GET /identity/v3/role_assignments?user.id=e4316a12a39f42a08812c00e07f343a0&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=007af6bd1b734517b934b4cac45faf2c => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:18.936343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08ce4b-1d2e-4f10-8dcf-9fb529b6063f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:18.936649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08ce4b-1d2e-4f10-8dcf-9fb529b6063f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:18.936921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08ce4b-1d2e-4f10-8dcf-9fb529b6063f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:18.940159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:18.966142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ef08ce4b-1d2e-4f10-8dcf-9fb529b6063f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:18.967615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ef08ce4b-1d2e-4f10-8dcf-9fb529b6063f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:18.970172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:19.013911 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceda4db-7e39-42a2-8f56-a2785a0c9f03 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:19.055277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 139/274] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:19.059081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:19.060633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.061128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.061644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.062634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:19.063134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:19.064788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:19.081257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 136/275] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:19.085821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08ce4b-1d2e-4f10-8dcf-9fb529b6063f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:19.087159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:19.201055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08ce4b-1d2e-4f10-8dcf-9fb529b6063f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:19.246983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a06f9-8774-480f-8608-a2ff04f721d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:19.248444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a06f9-8774-480f-8608-a2ff04f721d1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.248937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a06f9-8774-480f-8608-a2ff04f721d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.249224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a06f9-8774-480f-8608-a2ff04f721d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.250955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a06f9-8774-480f-8608-a2ff04f721d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:19.252588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a06f9-8774-480f-8608-a2ff04f721d1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:19.254936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 140/276] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:19.262324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a06f9-8774-480f-8608-a2ff04f721d1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:19.273069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786e740-c5e7-43f9-a0ed-091f56dcdde2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:19.274881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:19.292042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:19.313476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ef9fe-b91b-4d94-98a9-c04c46fdc821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:19.314359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef9fe-b91b-4d94-98a9-c04c46fdc821 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.314647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef9fe-b91b-4d94-98a9-c04c46fdc821 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.314927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef9fe-b91b-4d94-98a9-c04c46fdc821 None admin] PATH_INFO: `/v3/users/425c07a075184d4f9b2e3f2a025dbbee` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.331301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ef9fe-b91b-4d94-98a9-c04c46fdc821 None admin] RBAC: Authorizing `identity:get_user(user_id=425c07a075184d4f9b2e3f2a025dbbee)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:19.332796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ef9fe-b91b-4d94-98a9-c04c46fdc821 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:19.336652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 137/277] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:19 2022] GET /identity/v3/users/425c07a075184d4f9b2e3f2a025dbbee => generated 293 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:19.339038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c7484-efab-43a9-b1ae-9e33412b4a05 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.339038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c7484-efab-43a9-b1ae-9e33412b4a05 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.339038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c7484-efab-43a9-b1ae-9e33412b4a05 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.340152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 138/278] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:19.346714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c935-46db-461f-882a-675c8481654e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.346999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c935-46db-461f-882a-675c8481654e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.347947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c935-46db-461f-882a-675c8481654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.365956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e853c935-46db-461f-882a-675c8481654e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:19.367168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e853c935-46db-461f-882a-675c8481654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:19.384254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a06f9-8774-480f-8608-a2ff04f721d1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:19.426137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97d4b-d1a4-4247-965c-42986e08b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:19.426810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97d4b-d1a4-4247-965c-42986e08b0f9 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.427095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97d4b-d1a4-4247-965c-42986e08b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.427435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc97d4b-d1a4-4247-965c-42986e08b0f9 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.428164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97d4b-d1a4-4247-965c-42986e08b0f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:19.429613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc97d4b-d1a4-4247-965c-42986e08b0f9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:19.445535 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 139/279] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:19.446402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853c935-46db-461f-882a-675c8481654e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:19.524116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 141/280] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:19 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:19.524881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97d4b-d1a4-4247-965c-42986e08b0f9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:19.550969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:19.564337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853c935-46db-461f-882a-675c8481654e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:19.595160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:19.610967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ffe01-528c-4e4e-be3d-0db6ec73b103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:19.611786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ffe01-528c-4e4e-be3d-0db6ec73b103 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.612076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ffe01-528c-4e4e-be3d-0db6ec73b103 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.612355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ffe01-528c-4e4e-be3d-0db6ec73b103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.613213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ffe01-528c-4e4e-be3d-0db6ec73b103 None admin] RBAC: Adding query filter params (role.id=9d899c1b7b0740348cc8f312ed536187, user.id=425c07a075184d4f9b2e3f2a025dbbee, scope.system=all) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:19.613534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ffe01-528c-4e4e-be3d-0db6ec73b103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:19.615080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ffe01-528c-4e4e-be3d-0db6ec73b103 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:19.621330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 140/281] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 11 11:05:19 2022] GET /identity/v3/role_assignments?user.id=425c07a075184d4f9b2e3f2a025dbbee&scope.system=all&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:19.623217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424db5-274d-43b0-917d-ad023acf2202 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.623533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424db5-274d-43b0-917d-ad023acf2202 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.623816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424db5-274d-43b0-917d-ad023acf2202 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.627433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 141/282] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:19.638072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8377-8e55-410c-9308-ad6809171d81 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.638432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8377-8e55-410c-9308-ad6809171d81 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.638611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f8377-8e55-410c-9308-ad6809171d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.649090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc97d4b-d1a4-4247-965c-42986e08b0f9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:19.660608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f23f8377-8e55-410c-9308-ad6809171d81 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:19.662036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f23f8377-8e55-410c-9308-ad6809171d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:19.698470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160170d8-8a23-4941-b561-17ee81e24fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:19.699343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160170d8-8a23-4941-b561-17ee81e24fee None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.700569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160170d8-8a23-4941-b561-17ee81e24fee None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.700858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160170d8-8a23-4941-b561-17ee81e24fee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.714654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160170d8-8a23-4941-b561-17ee81e24fee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-160170d8-8a23-4941-b561-17ee81e24fee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:19.723441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:19.724423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:19.725892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 142/283] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:19.730894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160170d8-8a23-4941-b561-17ee81e24fee None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:19.754570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 142/284] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:19.779276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f8377-8e55-410c-9308-ad6809171d81 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:19.848514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160170d8-8a23-4941-b561-17ee81e24fee None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:19.875176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:19.891892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:19.892679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.892966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.893246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.894005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:19.894314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:19.896076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:19.901448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:19.906258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 143/285] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:19.912907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:19.913970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f8377-8e55-410c-9308-ad6809171d81 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:19.952608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037aaa0-82aa-45ca-851b-46d07baee3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:19.954120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037aaa0-82aa-45ca-851b-46d07baee3dc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:19.954120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037aaa0-82aa-45ca-851b-46d07baee3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:19.954120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037aaa0-82aa-45ca-851b-46d07baee3dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:19.969754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037aaa0-82aa-45ca-851b-46d07baee3dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5037aaa0-82aa-45ca-851b-46d07baee3dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:19.981010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:19.982027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:19.982843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:19.990305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 143/286] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:19.992312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037aaa0-82aa-45ca-851b-46d07baee3dc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:20.048947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ff768-3d9f-44b1-a830-49748019245a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:20.086942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bda28-5f61-4c37-8cc2-0ee93f6cd801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:20.087858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bda28-5f61-4c37-8cc2-0ee93f6cd801 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:20.088140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bda28-5f61-4c37-8cc2-0ee93f6cd801 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:20.088417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bda28-5f61-4c37-8cc2-0ee93f6cd801 None admin] PATH_INFO: `/v3/users/594b774f0bf048e9adde846098b8d1fc` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:20.089639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bda28-5f61-4c37-8cc2-0ee93f6cd801 None admin] RBAC: Authorizing `identity:get_user(user_id=594b774f0bf048e9adde846098b8d1fc)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:20.091864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bda28-5f61-4c37-8cc2-0ee93f6cd801 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:20.093906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 144/287] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:19 2022] GET /identity/v3/users/594b774f0bf048e9adde846098b8d1fc => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:20.100228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bda28-5f61-4c37-8cc2-0ee93f6cd801 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:20.125682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037aaa0-82aa-45ca-851b-46d07baee3dc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:20.167732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:20.209316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:20.210000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:20.210283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:20.210563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:20.211343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:20.211658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:20.213058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:20.218931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bda28-5f61-4c37-8cc2-0ee93f6cd801 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:20.245586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 144/288] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:20.252272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:20.260568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0b825-d08c-4238-bc5a-860e13f05594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:20.262354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0b825-d08c-4238-bc5a-860e13f05594 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:20.262692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0b825-d08c-4238-bc5a-860e13f05594 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:20.262977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0b825-d08c-4238-bc5a-860e13f05594 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:20.264241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0b825-d08c-4238-bc5a-860e13f05594 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:20.265695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0b825-d08c-4238-bc5a-860e13f05594 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:20.272762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 145/289] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:20 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:20.274030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0b825-d08c-4238-bc5a-860e13f05594 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:20.292049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:20.430129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6634ab-2b3c-4fb3-8937-d248d50f6eee None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:20.464780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a2bea-2267-42f8-a810-ee56d059c03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:20.465710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0b825-d08c-4238-bc5a-860e13f05594 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:20.465858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a2bea-2267-42f8-a810-ee56d059c03e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:20.466033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a2bea-2267-42f8-a810-ee56d059c03e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:20.466302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a2bea-2267-42f8-a810-ee56d059c03e None admin] PATH_INFO: `/v3/users/93ad569cd8264f20919c5d38c60ae507` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:20.467557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a2bea-2267-42f8-a810-ee56d059c03e None admin] RBAC: Authorizing `identity:get_user(user_id=93ad569cd8264f20919c5d38c60ae507)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:20.469086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a2bea-2267-42f8-a810-ee56d059c03e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:20.470957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 145/290] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:20 2022] GET /identity/v3/users/93ad569cd8264f20919c5d38c60ae507 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:20.480517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a2bea-2267-42f8-a810-ee56d059c03e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:20.503351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:20.533754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546412c-8af3-4a70-8d8f-305961c4a76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:20.534646 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546412c-8af3-4a70-8d8f-305961c4a76e None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:20.534981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546412c-8af3-4a70-8d8f-305961c4a76e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:20.535279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546412c-8af3-4a70-8d8f-305961c4a76e None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/users/594b774f0bf048e9adde846098b8d1fc/roles/3582abcd4fa246d4add3f71e7474643f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:20.538636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546412c-8af3-4a70-8d8f-305961c4a76e None admin] RBAC: Authorizing `identity:create_grant(project_id=bf653a9fe09a4a3e86a6e3e8c134711e, user_id=594b774f0bf048e9adde846098b8d1fc, role_id=3582abcd4fa246d4add3f71e7474643f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:20.540111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546412c-8af3-4a70-8d8f-305961c4a76e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:20.601622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:20.622085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 146/291] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:20 2022] PUT /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/users/594b774f0bf048e9adde846098b8d1fc/roles/3582abcd4fa246d4add3f71e7474643f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:05:20.633030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a2bea-2267-42f8-a810-ee56d059c03e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:20.672203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7baf5-5dc8-4527-928f-0ab4dc544e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:20.672873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7baf5-5dc8-4527-928f-0ab4dc544e88 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:20.673162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7baf5-5dc8-4527-928f-0ab4dc544e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:20.673443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7baf5-5dc8-4527-928f-0ab4dc544e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:20.749807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7baf5-5dc8-4527-928f-0ab4dc544e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:20.751227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7baf5-5dc8-4527-928f-0ab4dc544e88 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:20.836809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:20.858941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 146/292] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:20.862656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546412c-8af3-4a70-8d8f-305961c4a76e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:20.983982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546412c-8af3-4a70-8d8f-305961c4a76e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:21.035137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220b675-034e-4da9-ae40-72581adf2598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:21.036005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b675-034e-4da9-ae40-72581adf2598 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:21.036294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b675-034e-4da9-ae40-72581adf2598 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:21.036576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b675-034e-4da9-ae40-72581adf2598 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:21.037795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b675-034e-4da9-ae40-72581adf2598 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:21.039234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b675-034e-4da9-ae40-72581adf2598 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:21.041327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 147/293] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:20 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:21.047276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7baf5-5dc8-4527-928f-0ab4dc544e88 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:21.218040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7baf5-5dc8-4527-928f-0ab4dc544e88 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:21.266268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfedc61-77d1-4e4f-92aa-46a93adca176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:21.266933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfedc61-77d1-4e4f-92aa-46a93adca176 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:21.267222 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfedc61-77d1-4e4f-92aa-46a93adca176 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:21.267526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfedc61-77d1-4e4f-92aa-46a93adca176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:21.268363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfedc61-77d1-4e4f-92aa-46a93adca176 None admin] RBAC: Adding query filter params (role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e, user.id=93ad569cd8264f20919c5d38c60ae507) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:21.268682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfedc61-77d1-4e4f-92aa-46a93adca176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:21.270115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfedc61-77d1-4e4f-92aa-46a93adca176 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:21.276516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 147/294] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:21 2022] GET /identity/v3/role_assignments?user.id=93ad569cd8264f20919c5d38c60ae507&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:21.950681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b56f69-1c37-4ebd-a160-7360633891c3 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:21.951713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b56f69-1c37-4ebd-a160-7360633891c3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:21.952195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b56f69-1c37-4ebd-a160-7360633891c3 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:21.953790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 148/295] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:21.959814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7600-bca5-4626-9d8f-ef562a77fc16 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:21.960782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7600-bca5-4626-9d8f-ef562a77fc16 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:21.961253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7600-bca5-4626-9d8f-ef562a77fc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:21.982024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4afa7600-bca5-4626-9d8f-ef562a77fc16 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:21.983554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4afa7600-bca5-4626-9d8f-ef562a77fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:22.025774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:22.054734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 148/296] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:22.060885 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f068b-05d0-4e4f-8b25-1950c865f596 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.061184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f068b-05d0-4e4f-8b25-1950c865f596 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.061473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f068b-05d0-4e4f-8b25-1950c865f596 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.062844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 149/297] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:22.070118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671ed1b-4b25-437e-81ec-2ac92730ddee None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.070118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671ed1b-4b25-437e-81ec-2ac92730ddee None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.070118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671ed1b-4b25-437e-81ec-2ac92730ddee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.086539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2671ed1b-4b25-437e-81ec-2ac92730ddee None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:22.087152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f068b-05d0-4e4f-8b25-1950c865f596 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.087806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2671ed1b-4b25-437e-81ec-2ac92730ddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:22.144249 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:22.180288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 149/298] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:22.203650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671ed1b-4b25-437e-81ec-2ac92730ddee None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.204155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f068b-05d0-4e4f-8b25-1950c865f596 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.245040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de6bcc-7bc5-4355-9ff9-c90682da28a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:22.247602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6bcc-7bc5-4355-9ff9-c90682da28a5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.247880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6bcc-7bc5-4355-9ff9-c90682da28a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.248156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de6bcc-7bc5-4355-9ff9-c90682da28a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.265164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de6bcc-7bc5-4355-9ff9-c90682da28a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-95de6bcc-7bc5-4355-9ff9-c90682da28a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:22.277348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:22.278490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:22.279504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:22.286521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 150/299] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:22.287540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de6bcc-7bc5-4355-9ff9-c90682da28a5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.326043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671ed1b-4b25-437e-81ec-2ac92730ddee None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.328372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:22.362308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23a9a7-1c4b-4581-9b06-b672142e67a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:22.363110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23a9a7-1c4b-4581-9b06-b672142e67a3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.363460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23a9a7-1c4b-4581-9b06-b672142e67a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.363742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23a9a7-1c4b-4581-9b06-b672142e67a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.373630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23a9a7-1c4b-4581-9b06-b672142e67a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-9d23a9a7-1c4b-4581-9b06-b672142e67a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:22.381426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:22.382292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:22.383110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 150/300] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:22.383818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052bb2-b88b-4c15-9c0a-6932ce27192e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.384613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052bb2-b88b-4c15-9c0a-6932ce27192e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.384613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052bb2-b88b-4c15-9c0a-6932ce27192e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.385685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 151/301] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:22.390176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41052bb2-b88b-4c15-9c0a-6932ce27192e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.400327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de6bcc-7bc5-4355-9ff9-c90682da28a5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.410790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:22.436537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda9be7-f32a-42f4-af31-e7089ee00f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:22.437200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda9be7-f32a-42f4-af31-e7089ee00f92 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.437485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda9be7-f32a-42f4-af31-e7089ee00f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.437768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda9be7-f32a-42f4-af31-e7089ee00f92 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.438517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda9be7-f32a-42f4-af31-e7089ee00f92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:22.438841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda9be7-f32a-42f4-af31-e7089ee00f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:22.440232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda9be7-f32a-42f4-af31-e7089ee00f92 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:22.449468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 151/302] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:22.450682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414664a9-1cd0-4a98-b2e5-c002ad9aaa31 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.450971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414664a9-1cd0-4a98-b2e5-c002ad9aaa31 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.451271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414664a9-1cd0-4a98-b2e5-c002ad9aaa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.468242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-414664a9-1cd0-4a98-b2e5-c002ad9aaa31 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:22.469351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-414664a9-1cd0-4a98-b2e5-c002ad9aaa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:22.502407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41052bb2-b88b-4c15-9c0a-6932ce27192e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.540629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:22.541297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.541582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.541869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.551416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:22.551560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 152/303] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:22.551651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:22.551758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414664a9-1cd0-4a98-b2e5-c002ad9aaa31 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.551842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:22.554330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 152/304] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:22.559749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.588498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:22.653532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:22.656529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414664a9-1cd0-4a98-b2e5-c002ad9aaa31 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.665982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f25f6-af96-4542-8c49-02b5ba7e5862 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.694743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cd687-230b-44b7-b075-69b3ef3ef0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:22.696146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd687-230b-44b7-b075-69b3ef3ef0ac None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.696560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd687-230b-44b7-b075-69b3ef3ef0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.696840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cd687-230b-44b7-b075-69b3ef3ef0ac None admin] PATH_INFO: `/v3/users/4ce066f8a6f645e48f55002da1a71552` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.703406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec621c2-a39a-4250-ad89-20e2ef7293dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:22.704280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec621c2-a39a-4250-ad89-20e2ef7293dd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.704568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec621c2-a39a-4250-ad89-20e2ef7293dd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.705239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec621c2-a39a-4250-ad89-20e2ef7293dd None admin] PATH_INFO: `/v3/users/425c07a075184d4f9b2e3f2a025dbbee` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.706445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec621c2-a39a-4250-ad89-20e2ef7293dd None admin] RBAC: Authorizing `identity:get_user(user_id=425c07a075184d4f9b2e3f2a025dbbee)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:22.707917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec621c2-a39a-4250-ad89-20e2ef7293dd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:22.709534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 153/305] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:22 2022] GET /identity/v3/users/425c07a075184d4f9b2e3f2a025dbbee => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:22.714180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec621c2-a39a-4250-ad89-20e2ef7293dd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.714667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cd687-230b-44b7-b075-69b3ef3ef0ac None admin] RBAC: Authorizing `identity:get_user(user_id=4ce066f8a6f645e48f55002da1a71552)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:22.716146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cd687-230b-44b7-b075-69b3ef3ef0ac None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:22.717944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 153/306] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:22 2022] GET /identity/v3/users/4ce066f8a6f645e48f55002da1a71552 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:22.718683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cd687-230b-44b7-b075-69b3ef3ef0ac None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.816800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:22.823851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec621c2-a39a-4250-ad89-20e2ef7293dd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.831927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cd687-230b-44b7-b075-69b3ef3ef0ac None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.861211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e9bfc-5c12-4b9b-a5c8-025bc2f60cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:22.862151 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e9bfc-5c12-4b9b-a5c8-025bc2f60cfc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.862741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e9bfc-5c12-4b9b-a5c8-025bc2f60cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.863157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e9bfc-5c12-4b9b-a5c8-025bc2f60cfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.864438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e9bfc-5c12-4b9b-a5c8-025bc2f60cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:22.865882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77e9bfc-5c12-4b9b-a5c8-025bc2f60cfc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:22.866920 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a115680-a506-431a-8f77-7c9bc9629436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:22.872538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a115680-a506-431a-8f77-7c9bc9629436 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:22.872673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 154/307] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:22.872753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a115680-a506-431a-8f77-7c9bc9629436 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:22.872753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a115680-a506-431a-8f77-7c9bc9629436 None admin] PATH_INFO: `/v3/system/users/425c07a075184d4f9b2e3f2a025dbbee/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:22.872885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e9bfc-5c12-4b9b-a5c8-025bc2f60cfc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:22.873259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a115680-a506-431a-8f77-7c9bc9629436 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=425c07a075184d4f9b2e3f2a025dbbee, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:22.874812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a115680-a506-431a-8f77-7c9bc9629436 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:22.968005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e9bfc-5c12-4b9b-a5c8-025bc2f60cfc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:22.987652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 154/308] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Jul 11 11:05:22 2022] PUT /identity/v3/system/users/425c07a075184d4f9b2e3f2a025dbbee/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:22.987652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a115680-a506-431a-8f77-7c9bc9629436 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:23.005086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86e177-da8d-40b3-a87e-4069291a6119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:23.005842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86e177-da8d-40b3-a87e-4069291a6119 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:23.006143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86e177-da8d-40b3-a87e-4069291a6119 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:23.006412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86e177-da8d-40b3-a87e-4069291a6119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:23.007293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86e177-da8d-40b3-a87e-4069291a6119 None admin] RBAC: Adding query filter params (role.id=3582abcd4fa246d4add3f71e7474643f, user.id=4ce066f8a6f645e48f55002da1a71552, scope.system=all) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:23.007614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86e177-da8d-40b3-a87e-4069291a6119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:23.009121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86e177-da8d-40b3-a87e-4069291a6119 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:23.023290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 155/309] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 11 11:05:22 2022] GET /identity/v3/role_assignments?user.id=4ce066f8a6f645e48f55002da1a71552&scope.system=all&role.id=3582abcd4fa246d4add3f71e7474643f => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:23.031700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:23.073091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecafd6-9055-4af9-a4de-49f41c25f847 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:23.073384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecafd6-9055-4af9-a4de-49f41c25f847 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:23.073667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecafd6-9055-4af9-a4de-49f41c25f847 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:23.075298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 156/310] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:23.083307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0196-90db-4220-a983-f2d1f1646199 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:23.083869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0196-90db-4220-a983-f2d1f1646199 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:23.084335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0196-90db-4220-a983-f2d1f1646199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:23.111409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6f5d0196-90db-4220-a983-f2d1f1646199 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:23.112883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6f5d0196-90db-4220-a983-f2d1f1646199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:23.138893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:23.139939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a115680-a506-431a-8f77-7c9bc9629436 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:23.187487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f1575-7d10-4970-bf74-6f7a3d08bf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:23.188629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1575-7d10-4970-bf74-6f7a3d08bf00 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:23.188629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1575-7d10-4970-bf74-6f7a3d08bf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:23.188807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1575-7d10-4970-bf74-6f7a3d08bf00 None admin] PATH_INFO: `/v3/groups` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:23.189425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f1575-7d10-4970-bf74-6f7a3d08bf00 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:23.190912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f1575-7d10-4970-bf74-6f7a3d08bf00 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:23.199903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 157/311] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:23.223677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0196-90db-4220-a983-f2d1f1646199 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:23.261608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 155/312] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Jul 11 11:05:22 2022] POST /identity/v3/groups => generated 229 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:23.351327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0196-90db-4220-a983-f2d1f1646199 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:23.389664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d59b2-93e4-4c54-9fc6-8a94e18998a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:23.390477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d59b2-93e4-4c54-9fc6-8a94e18998a1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:23.390764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d59b2-93e4-4c54-9fc6-8a94e18998a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:23.391036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d59b2-93e4-4c54-9fc6-8a94e18998a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:23.403033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d59b2-93e4-4c54-9fc6-8a94e18998a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-157d59b2-93e4-4c54-9fc6-8a94e18998a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:23.411041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:23.412016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:23.412830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 158/313] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:23.419856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f1575-7d10-4970-bf74-6f7a3d08bf00 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:23.504976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:23.524951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f1575-7d10-4970-bf74-6f7a3d08bf00 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:23.559563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:23.560238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:23.560522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:23.560849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:23.561611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:23.561921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:23.563462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:23.572701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 156/314] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:23.579060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d59b2-93e4-4c54-9fc6-8a94e18998a1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:23.598078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:23.684786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d59b2-93e4-4c54-9fc6-8a94e18998a1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:23.722389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2d412-f681-4b67-935f-7730717d91bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:23.723270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2d412-f681-4b67-935f-7730717d91bd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:23.723748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2d412-f681-4b67-935f-7730717d91bd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:23.724028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2d412-f681-4b67-935f-7730717d91bd None admin] PATH_INFO: `/v3/users/594b774f0bf048e9adde846098b8d1fc` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:23.725227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2d412-f681-4b67-935f-7730717d91bd None admin] RBAC: Authorizing `identity:get_user(user_id=594b774f0bf048e9adde846098b8d1fc)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:23.726661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2d412-f681-4b67-935f-7730717d91bd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:23.728298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 159/315] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:23 2022] GET /identity/v3/users/594b774f0bf048e9adde846098b8d1fc => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:23.733822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:23.846240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53125d85-8d0d-4cae-8102-2f54911138d4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:23.879809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5024546-cbd3-4f21-a30b-194122afe0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:23.880453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5024546-cbd3-4f21-a30b-194122afe0cc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:23.880735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5024546-cbd3-4f21-a30b-194122afe0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:23.881010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5024546-cbd3-4f21-a30b-194122afe0cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:23.899585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:23.949263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5024546-cbd3-4f21-a30b-194122afe0cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:23.950761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5024546-cbd3-4f21-a30b-194122afe0cc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:24.049702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 157/316] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:24.055486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2d412-f681-4b67-935f-7730717d91bd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:24.141509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:24.157300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2d412-f681-4b67-935f-7730717d91bd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:24.195369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d447f7b-b9e2-4406-8d4d-268f3ccaa684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:24.196253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d447f7b-b9e2-4406-8d4d-268f3ccaa684 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:24.196535 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d447f7b-b9e2-4406-8d4d-268f3ccaa684 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:24.196812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d447f7b-b9e2-4406-8d4d-268f3ccaa684 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:24.198332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d447f7b-b9e2-4406-8d4d-268f3ccaa684 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:24.200768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d447f7b-b9e2-4406-8d4d-268f3ccaa684 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:24.206261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 160/317] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:24 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:24.208798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5024546-cbd3-4f21-a30b-194122afe0cc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:24.261097 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:24.308790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5024546-cbd3-4f21-a30b-194122afe0cc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:24.342912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d414e07-67de-4c72-bf2b-2c1aa4522753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:24.343598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e07-67de-4c72-bf2b-2c1aa4522753 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:24.343878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e07-67de-4c72-bf2b-2c1aa4522753 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:24.344152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e07-67de-4c72-bf2b-2c1aa4522753 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:24.344987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d414e07-67de-4c72-bf2b-2c1aa4522753 None admin] RBAC: Adding query filter params (role.id=3582abcd4fa246d4add3f71e7474643f, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e, user.id=594b774f0bf048e9adde846098b8d1fc) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:24.345355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d414e07-67de-4c72-bf2b-2c1aa4522753 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:24.346837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d414e07-67de-4c72-bf2b-2c1aa4522753 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:24.353201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 158/318] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:24 2022] GET /identity/v3/role_assignments?user.id=594b774f0bf048e9adde846098b8d1fc&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=3582abcd4fa246d4add3f71e7474643f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:25.116198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e6ad-25ca-44e7-bf41-9789e8f66e61 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.116760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e6ad-25ca-44e7-bf41-9789e8f66e61 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.116853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0e6ad-25ca-44e7-bf41-9789e8f66e61 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.118484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 161/319] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:25.118694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0fc61-dc02-4008-b6ac-7816ba6e879f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.119099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0fc61-dc02-4008-b6ac-7816ba6e879f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.119282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0fc61-dc02-4008-b6ac-7816ba6e879f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.120715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 159/320] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:25.123445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e738-c2ca-4cfd-bc97-6fe752da4236 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.123737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e738-c2ca-4cfd-bc97-6fe752da4236 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.124019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e738-c2ca-4cfd-bc97-6fe752da4236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.125573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41090b3a-7eec-41c6-b011-d8c0ad88a6fe None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.125974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41090b3a-7eec-41c6-b011-d8c0ad88a6fe None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.126254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41090b3a-7eec-41c6-b011-d8c0ad88a6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.143033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-61d6e738-c2ca-4cfd-bc97-6fe752da4236 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:25.143230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-41090b3a-7eec-41c6-b011-d8c0ad88a6fe None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:25.144265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-61d6e738-c2ca-4cfd-bc97-6fe752da4236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:25.144389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-41090b3a-7eec-41c6-b011-d8c0ad88a6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:25.217525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 160/321] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:25.222344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 162/322] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:25.239598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41090b3a-7eec-41c6-b011-d8c0ad88a6fe None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.245753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6e738-c2ca-4cfd-bc97-6fe752da4236 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.291351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:25.342951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:25.354306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41090b3a-7eec-41c6-b011-d8c0ad88a6fe None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.361317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6e738-c2ca-4cfd-bc97-6fe752da4236 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.389398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e47e2a-4de8-48a6-b63f-67ec4459ed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.390191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e47e2a-4de8-48a6-b63f-67ec4459ed50 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.390476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e47e2a-4de8-48a6-b63f-67ec4459ed50 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.390751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e47e2a-4de8-48a6-b63f-67ec4459ed50 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.396724 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61f843-40cf-4178-a520-a94b22ed6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.397510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61f843-40cf-4178-a520-a94b22ed6a84 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.397795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61f843-40cf-4178-a520-a94b22ed6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.398070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61f843-40cf-4178-a520-a94b22ed6a84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.401482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e47e2a-4de8-48a6-b63f-67ec4459ed50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:25.408440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61f843-40cf-4178-a520-a94b22ed6a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-82e47e2a-4de8-48a6-b63f-67ec4459ed50 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:25.409760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:25.410838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 161/323] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:25.412183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8eae24-8517-4960-aa76-a376395a1187 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.413171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8eae24-8517-4960-aa76-a376395a1187 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.413171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8eae24-8517-4960-aa76-a376395a1187 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.414091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 162/324] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-9b61f843-40cf-4178-a520-a94b22ed6a84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:25.416114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:25.417166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:25.424201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 163/325] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:25.425504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8eae24-8517-4960-aa76-a376395a1187 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.426356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45606afb-5fe1-4e23-bf75-ed32932c5eb5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.426356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45606afb-5fe1-4e23-bf75-ed32932c5eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.426356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45606afb-5fe1-4e23-bf75-ed32932c5eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.436853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-45606afb-5fe1-4e23-bf75-ed32932c5eb5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:25.438042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-45606afb-5fe1-4e23-bf75-ed32932c5eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:25.503937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 164/326] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:25.504760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45606afb-5fe1-4e23-bf75-ed32932c5eb5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.515792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:25.526582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8eae24-8517-4960-aa76-a376395a1187 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.561093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.561750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.562030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.562307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.563055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:25.563393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:25.564766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:25.571297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:25.572585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 163/327] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:25.573367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.600882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45606afb-5fe1-4e23-bf75-ed32932c5eb5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.634607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.635290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.635588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.635868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.636615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:25.636925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:25.638839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:25.646789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 165/328] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:25.647603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.673806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20a24f-9595-4a12-929f-8ac76eee67d8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.706528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57a36-4828-43e6-9938-4cd4f22f969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.707320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57a36-4828-43e6-9938-4cd4f22f969c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.707700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57a36-4828-43e6-9938-4cd4f22f969c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.707982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57a36-4828-43e6-9938-4cd4f22f969c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.709152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af57a36-4828-43e6-9938-4cd4f22f969c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:25.710594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af57a36-4828-43e6-9938-4cd4f22f969c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:25.712229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 164/329] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:25.712980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57a36-4828-43e6-9938-4cd4f22f969c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.740595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:25.748070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cf3d5-8af3-49f9-857a-b9b1c81b1e19 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.780351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cacc7-40de-4cda-bd1f-a44b997208e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.781253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cacc7-40de-4cda-bd1f-a44b997208e1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.781536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cacc7-40de-4cda-bd1f-a44b997208e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.781813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cacc7-40de-4cda-bd1f-a44b997208e1 None admin] PATH_INFO: `/v3/users/4ce066f8a6f645e48f55002da1a71552` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.783006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627cacc7-40de-4cda-bd1f-a44b997208e1 None admin] RBAC: Authorizing `identity:get_user(user_id=4ce066f8a6f645e48f55002da1a71552)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:25.784488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627cacc7-40de-4cda-bd1f-a44b997208e1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:25.786218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 166/330] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:25 2022] GET /identity/v3/users/4ce066f8a6f645e48f55002da1a71552 => generated 293 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:25.786952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cacc7-40de-4cda-bd1f-a44b997208e1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.789409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:25.815273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57a36-4828-43e6-9938-4cd4f22f969c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.848939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df91b63-1970-4359-a422-afb5a80a9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.849798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df91b63-1970-4359-a422-afb5a80a9a6d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.850080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df91b63-1970-4359-a422-afb5a80a9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.850353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df91b63-1970-4359-a422-afb5a80a9a6d None admin] PATH_INFO: `/v3/users/425c07a075184d4f9b2e3f2a025dbbee` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.851620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df91b63-1970-4359-a422-afb5a80a9a6d None admin] RBAC: Authorizing `identity:get_user(user_id=425c07a075184d4f9b2e3f2a025dbbee)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:25.852984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df91b63-1970-4359-a422-afb5a80a9a6d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:25.854602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 165/331] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:25 2022] GET /identity/v3/users/425c07a075184d4f9b2e3f2a025dbbee => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:25.855354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df91b63-1970-4359-a422-afb5a80a9a6d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:25.884126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cacc7-40de-4cda-bd1f-a44b997208e1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.919147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3840b4-4aa0-43a0-be7f-995888aa513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.919827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3840b4-4aa0-43a0-be7f-995888aa513f None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.920116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3840b4-4aa0-43a0-be7f-995888aa513f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.920392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3840b4-4aa0-43a0-be7f-995888aa513f None admin] PATH_INFO: `/v3/groups` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.921073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3840b4-4aa0-43a0-be7f-995888aa513f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:25.922490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3840b4-4aa0-43a0-be7f-995888aa513f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:25.948315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:25.954577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df91b63-1970-4359-a422-afb5a80a9a6d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:25.986601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31752cae-7179-4c32-8443-3207b1b763e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:25.987477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31752cae-7179-4c32-8443-3207b1b763e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:25.987766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31752cae-7179-4c32-8443-3207b1b763e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:25.988047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31752cae-7179-4c32-8443-3207b1b763e6 None admin] PATH_INFO: `/v3/system/users/4ce066f8a6f645e48f55002da1a71552/roles/3582abcd4fa246d4add3f71e7474643f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:25.989523 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31752cae-7179-4c32-8443-3207b1b763e6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4ce066f8a6f645e48f55002da1a71552, role_id=3582abcd4fa246d4add3f71e7474643f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:25.990885 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31752cae-7179-4c32-8443-3207b1b763e6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:26.018216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 167/332] 10.222.0.26 () {60 vars in 1134 bytes} [Mon Jul 11 11:05:25 2022] POST /identity/v3/groups => generated 226 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:26.019201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3840b4-4aa0-43a0-be7f-995888aa513f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:26.085315 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:26.087974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 166/333] 10.222.0.26 () {58 vars in 1334 bytes} [Mon Jul 11 11:05:25 2022] PUT /identity/v3/system/users/4ce066f8a6f645e48f55002da1a71552/roles/3582abcd4fa246d4add3f71e7474643f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:26.132149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3840b4-4aa0-43a0-be7f-995888aa513f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:26.173347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b22229-c145-49d2-9815-55faee2a35e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:26.174120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22229-c145-49d2-9815-55faee2a35e0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:26.174408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22229-c145-49d2-9815-55faee2a35e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:26.174688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22229-c145-49d2-9815-55faee2a35e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:26.175849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b22229-c145-49d2-9815-55faee2a35e0 None admin] RBAC: Adding query filter params (role.id=9d899c1b7b0740348cc8f312ed536187, user.id=425c07a075184d4f9b2e3f2a025dbbee, scope.system=all) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:26.176171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b22229-c145-49d2-9815-55faee2a35e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:26.178933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b22229-c145-49d2-9815-55faee2a35e0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:26.194692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 168/334] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 11 11:05:26 2022] GET /identity/v3/role_assignments?user.id=425c07a075184d4f9b2e3f2a025dbbee&scope.system=all&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:28.470323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8299e4-f2dd-4cb4-a857-0ab9c0af0c50 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:28.470877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8299e4-f2dd-4cb4-a857-0ab9c0af0c50 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:28.470877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8299e4-f2dd-4cb4-a857-0ab9c0af0c50 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:28.472565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 167/335] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:28.484769 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dcfe7-c394-4aff-a8a8-0dfe7504ff5f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:28.485704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dcfe7-c394-4aff-a8a8-0dfe7504ff5f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:28.486206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002dcfe7-c394-4aff-a8a8-0dfe7504ff5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:28.508550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-002dcfe7-c394-4aff-a8a8-0dfe7504ff5f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:28.510094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-002dcfe7-c394-4aff-a8a8-0dfe7504ff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:28.549173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32cb9c-9c3f-46c3-b355-ab5c85af07cc None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:28.549386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32cb9c-9c3f-46c3-b355-ab5c85af07cc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:28.549588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32cb9c-9c3f-46c3-b355-ab5c85af07cc None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:28.551204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 168/336] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:28.557464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ebf75-dfd7-4eb2-a227-b05e559e0516 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:28.557761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ebf75-dfd7-4eb2-a227-b05e559e0516 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:28.558050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ebf75-dfd7-4eb2-a227-b05e559e0516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:28.588712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-df5ebf75-dfd7-4eb2-a227-b05e559e0516 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:28.589988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-df5ebf75-dfd7-4eb2-a227-b05e559e0516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:28.594791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 169/337] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:28.630609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dcfe7-c394-4aff-a8a8-0dfe7504ff5f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:28.704187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:28.735359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 169/338] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:28.773351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ebf75-dfd7-4eb2-a227-b05e559e0516 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:28.775493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002dcfe7-c394-4aff-a8a8-0dfe7504ff5f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:28.817738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671f324-726c-4371-81f9-d4c275923f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:28.818906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671f324-726c-4371-81f9-d4c275923f3d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:28.819463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671f324-726c-4371-81f9-d4c275923f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:28.819955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671f324-726c-4371-81f9-d4c275923f3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:28.831911 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671f324-726c-4371-81f9-d4c275923f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1671f324-726c-4371-81f9-d4c275923f3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:28.840110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:28.841139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:28.841942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:28.844349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 170/339] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:28.852643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb203c-773a-4788-b3aa-904776a180e1 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:28.852643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb203c-773a-4788-b3aa-904776a180e1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:28.852643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb203c-773a-4788-b3aa-904776a180e1 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:28.852643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 171/340] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:28.853100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb203c-773a-4788-b3aa-904776a180e1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:28.957407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:28.970580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ebf75-dfd7-4eb2-a227-b05e559e0516 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:29.007627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb203c-773a-4788-b3aa-904776a180e1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:29.019882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:29.030027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba8db6-4dc9-4e4c-b79c-1484ac8bed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:29.031130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba8db6-4dc9-4e4c-b79c-1484ac8bed71 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.032641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba8db6-4dc9-4e4c-b79c-1484ac8bed71 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.033105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba8db6-4dc9-4e4c-b79c-1484ac8bed71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.062981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba8db6-4dc9-4e4c-b79c-1484ac8bed71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f0ba8db6-4dc9-4e4c-b79c-1484ac8bed71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:29.077478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:29.078369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 170/341] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:29.079196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5a4d-52c5-4c8a-8913-87522c28f9eb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.079980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5a4d-52c5-4c8a-8913-87522c28f9eb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.079980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f5a4d-52c5-4c8a-8913-87522c28f9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.081405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c711d-580a-4e59-99e2-e4e9c16cd6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:29.082325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c711d-580a-4e59-99e2-e4e9c16cd6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.082819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c711d-580a-4e59-99e2-e4e9c16cd6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.083347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c711d-580a-4e59-99e2-e4e9c16cd6d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.084428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c711d-580a-4e59-99e2-e4e9c16cd6d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:29.084941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c711d-580a-4e59-99e2-e4e9c16cd6d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:29.086731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35c711d-580a-4e59-99e2-e4e9c16cd6d3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:29.097189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 172/342] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:29.098776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e119d0-2c64-437a-9327-aca341b7083f None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.099337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e119d0-2c64-437a-9327-aca341b7083f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.099846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e119d0-2c64-437a-9327-aca341b7083f None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.101302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 173/343] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:29.102194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e119d0-2c64-437a-9327-aca341b7083f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:29.112577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-272f5a4d-52c5-4c8a-8913-87522c28f9eb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:29.117445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-272f5a4d-52c5-4c8a-8913-87522c28f9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:29.219557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e119d0-2c64-437a-9327-aca341b7083f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:29.221213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 171/344] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:29.228031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f5a4d-52c5-4c8a-8913-87522c28f9eb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:29.258481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852df2bd-f532-4747-9cb6-d0999adc15ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:29.259403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852df2bd-f532-4747-9cb6-d0999adc15ce None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.259978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852df2bd-f532-4747-9cb6-d0999adc15ce None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.260391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852df2bd-f532-4747-9cb6-d0999adc15ce None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.261687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852df2bd-f532-4747-9cb6-d0999adc15ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:29.262188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852df2bd-f532-4747-9cb6-d0999adc15ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:29.263744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852df2bd-f532-4747-9cb6-d0999adc15ce None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:29.273282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 174/345] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:29.275047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905685e3-e3f9-4bcb-948b-2186f8398b11 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.275504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905685e3-e3f9-4bcb-948b-2186f8398b11 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.275918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905685e3-e3f9-4bcb-948b-2186f8398b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.302648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-905685e3-e3f9-4bcb-948b-2186f8398b11 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:29.310795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-905685e3-e3f9-4bcb-948b-2186f8398b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:29.327693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:29.335810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:29.350327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f5a4d-52c5-4c8a-8913-87522c28f9eb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:29.382923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 175/346] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:29.384142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905685e3-e3f9-4bcb-948b-2186f8398b11 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:29.391734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fa40a-e39b-4307-b2f6-6a7e1d9288ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:29.392411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa40a-e39b-4307-b2f6-6a7e1d9288ea None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.392696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa40a-e39b-4307-b2f6-6a7e1d9288ea None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.392974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa40a-e39b-4307-b2f6-6a7e1d9288ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.466672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fa40a-e39b-4307-b2f6-6a7e1d9288ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:29.468420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fa40a-e39b-4307-b2f6-6a7e1d9288ea None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:29.497245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905685e3-e3f9-4bcb-948b-2186f8398b11 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:29.534519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d5d27-a795-46a7-aad0-d1ed647d122b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:29.536660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d5d27-a795-46a7-aad0-d1ed647d122b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.537246 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d5d27-a795-46a7-aad0-d1ed647d122b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.537741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d5d27-a795-46a7-aad0-d1ed647d122b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.552049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:29.572518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 172/347] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:29.573301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fa40a-e39b-4307-b2f6-6a7e1d9288ea None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:29.573628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:29.620821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d5d27-a795-46a7-aad0-d1ed647d122b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:29.622943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d5d27-a795-46a7-aad0-d1ed647d122b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:29.676390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fa40a-e39b-4307-b2f6-6a7e1d9288ea None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:29.712431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7bfcc-b5f7-4194-8c98-e0fbc6e1940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:29.713228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7bfcc-b5f7-4194-8c98-e0fbc6e1940f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.713517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7bfcc-b5f7-4194-8c98-e0fbc6e1940f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.713788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7bfcc-b5f7-4194-8c98-e0fbc6e1940f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.725957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7bfcc-b5f7-4194-8c98-e0fbc6e1940f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-58b7bfcc-b5f7-4194-8c98-e0fbc6e1940f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:29.735410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:29.736413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:29.737233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 173/349] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:29.737914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7bfcc-b5f7-4194-8c98-e0fbc6e1940f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:29.738566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 176/348] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:29.739748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d5d27-a795-46a7-aad0-d1ed647d122b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:29.783329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:29.808410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:29.847927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7bfcc-b5f7-4194-8c98-e0fbc6e1940f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:29.849736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d5d27-a795-46a7-aad0-d1ed647d122b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:29.886946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff23341-f021-44a7-92c3-3697b4c7f915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:29.887805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff23341-f021-44a7-92c3-3697b4c7f915 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.888089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff23341-f021-44a7-92c3-3697b4c7f915 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.888363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff23341-f021-44a7-92c3-3697b4c7f915 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.889511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847afaac-20c2-4b64-bdec-902c50905257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:29.890685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847afaac-20c2-4b64-bdec-902c50905257 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:29.891244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847afaac-20c2-4b64-bdec-902c50905257 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:29.891738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847afaac-20c2-4b64-bdec-902c50905257 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:29.893152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847afaac-20c2-4b64-bdec-902c50905257 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:29.894813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847afaac-20c2-4b64-bdec-902c50905257 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:29.897279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 177/350] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:29 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:29.898265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847afaac-20c2-4b64-bdec-902c50905257 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:29.903272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff23341-f021-44a7-92c3-3697b4c7f915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-2ff23341-f021-44a7-92c3-3697b4c7f915 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:29.908367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:29.909266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:29.910064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 174/351] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:29.911217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff23341-f021-44a7-92c3-3697b4c7f915 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.012874 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847afaac-20c2-4b64-bdec-902c50905257 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.027073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:30.031196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff23341-f021-44a7-92c3-3697b4c7f915 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.062662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:30.070291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7b636-5f91-448c-bb2d-08f3b06f548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.071219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b636-5f91-448c-bb2d-08f3b06f548b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.071569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b636-5f91-448c-bb2d-08f3b06f548b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.071846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7b636-5f91-448c-bb2d-08f3b06f548b None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.073403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7b636-5f91-448c-bb2d-08f3b06f548b None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.075012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7b636-5f91-448c-bb2d-08f3b06f548b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.077208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.077875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.078158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.078441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.080477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:30.080605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 178/352] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:29 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:30.080689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.080882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.084256 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7b636-5f91-448c-bb2d-08f3b06f548b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.093848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 175/353] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:30.094698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.211835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2a713-d657-478b-a58b-37f48ec5b7fd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.221259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7b636-5f91-448c-bb2d-08f3b06f548b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.258308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439041a5-0450-4532-936d-2d601ce3c858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.259668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439041a5-0450-4532-936d-2d601ce3c858 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.259668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439041a5-0450-4532-936d-2d601ce3c858 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.259668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439041a5-0450-4532-936d-2d601ce3c858 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.260311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439041a5-0450-4532-936d-2d601ce3c858 None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:30.260730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439041a5-0450-4532-936d-2d601ce3c858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.261154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564290f-8215-422c-8888-384f27609871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.262050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564290f-8215-422c-8888-384f27609871 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.262149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439041a5-0450-4532-936d-2d601ce3c858 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.262333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564290f-8215-422c-8888-384f27609871 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.262607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564290f-8215-422c-8888-384f27609871 None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.273558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 179/354] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:30.278213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564290f-8215-422c-8888-384f27609871 None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.278213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0564290f-8215-422c-8888-384f27609871 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.279435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb15cf-cee7-4d0e-8fe8-eb1ffb6e75e8 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.279766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 176/355] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:30.279900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb15cf-cee7-4d0e-8fe8-eb1ffb6e75e8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.280047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb15cf-cee7-4d0e-8fe8-eb1ffb6e75e8 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.280819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564290f-8215-422c-8888-384f27609871 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.281445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 180/356] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:30.282275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb15cf-cee7-4d0e-8fe8-eb1ffb6e75e8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.316339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:30.364164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:30.412393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564290f-8215-422c-8888-384f27609871 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.413577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb15cf-cee7-4d0e-8fe8-eb1ffb6e75e8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.452634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f592bb-2269-47b7-951b-c5c318b991c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.452833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70321faf-8474-4154-a781-efb3b702c741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.453114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f592bb-2269-47b7-951b-c5c318b991c0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.453319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f592bb-2269-47b7-951b-c5c318b991c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.453475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321faf-8474-4154-a781-efb3b702c741 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.453628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f592bb-2269-47b7-951b-c5c318b991c0 None admin] PATH_INFO: `/v3/users/4ce066f8a6f645e48f55002da1a71552` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.453724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321faf-8474-4154-a781-efb3b702c741 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.453992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321faf-8474-4154-a781-efb3b702c741 None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.455197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321faf-8474-4154-a781-efb3b702c741 None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.456074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f592bb-2269-47b7-951b-c5c318b991c0 None admin] RBAC: Authorizing `identity:get_user(user_id=4ce066f8a6f645e48f55002da1a71552)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.456627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321faf-8474-4154-a781-efb3b702c741 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.457493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f592bb-2269-47b7-951b-c5c318b991c0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.459116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 181/357] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/users/4ce066f8a6f645e48f55002da1a71552 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:30.463283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 177/358] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:30.463414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f592bb-2269-47b7-951b-c5c318b991c0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.463491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e0558-9656-4487-8e85-57105e20a3cd None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.463491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e0558-9656-4487-8e85-57105e20a3cd None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.463491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e0558-9656-4487-8e85-57105e20a3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.481046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-172e0558-9656-4487-8e85-57105e20a3cd None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:30.482246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-172e0558-9656-4487-8e85-57105e20a3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:30.563616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 178/359] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:30.572067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e0558-9656-4487-8e85-57105e20a3cd None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.583192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f592bb-2269-47b7-951b-c5c318b991c0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.601898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:30.615954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:30.623719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c902a-aae5-451c-a658-6e47f0386ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.623719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c902a-aae5-451c-a658-6e47f0386ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.624050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c902a-aae5-451c-a658-6e47f0386ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.624513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c902a-aae5-451c-a658-6e47f0386ad8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.686978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e0558-9656-4487-8e85-57105e20a3cd None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.705053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c902a-aae5-451c-a658-6e47f0386ad8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.706696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c902a-aae5-451c-a658-6e47f0386ad8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.724841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1707c-4051-4e7c-b6ca-a56ab32736f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.725496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1707c-4051-4e7c-b6ca-a56ab32736f5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.725781 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1707c-4051-4e7c-b6ca-a56ab32736f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.726052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1707c-4051-4e7c-b6ca-a56ab32736f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.726890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1707c-4051-4e7c-b6ca-a56ab32736f5 None admin] RBAC: Adding query filter params (group.id=197daa5d6e3045e9beffe561914b2d9c, role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=6ca74ec2a86e48a28a003d344862f755) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:30.727202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1707c-4051-4e7c-b6ca-a56ab32736f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.730515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1707c-4051-4e7c-b6ca-a56ab32736f5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.736522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 179/360] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/role_assignments?group.id=197daa5d6e3045e9beffe561914b2d9c&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:30.737779 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e801e1-5265-43e2-8751-d4743fd36553 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.738066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e801e1-5265-43e2-8751-d4743fd36553 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.738344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e801e1-5265-43e2-8751-d4743fd36553 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.740249 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 180/361] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:30.740394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e801e1-5265-43e2-8751-d4743fd36553 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.849809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 182/362] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:30.850610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c902a-aae5-451c-a658-6e47f0386ad8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.872004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:30.889020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:30.890749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e801e1-5265-43e2-8751-d4743fd36553 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:30.938723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:30.939425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:30.939767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:30.940045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:30.941485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] RBAC: Adding query filter params (role.id=3582abcd4fa246d4add3f71e7474643f, user.id=4ce066f8a6f645e48f55002da1a71552, scope.system=all) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:30.941801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:30.944192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:30.957837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 181/363] 10.222.0.26 () {56 vars in 1308 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/role_assignments?user.id=4ce066f8a6f645e48f55002da1a71552&scope.system=all&role.id=3582abcd4fa246d4add3f71e7474643f => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:30.958675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:30.969079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c902a-aae5-451c-a658-6e47f0386ad8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:31.012742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c511e7f-cd2c-4b54-b588-c4e937cb02c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:31.013417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c511e7f-cd2c-4b54-b588-c4e937cb02c2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.013703 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c511e7f-cd2c-4b54-b588-c4e937cb02c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.013980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c511e7f-cd2c-4b54-b588-c4e937cb02c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.015389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c511e7f-cd2c-4b54-b588-c4e937cb02c2 None admin] RBAC: Adding query filter params (group.id=197daa5d6e3045e9beffe561914b2d9c, role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=6ca74ec2a86e48a28a003d344862f755) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:31.015751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c511e7f-cd2c-4b54-b588-c4e937cb02c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:31.017237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c511e7f-cd2c-4b54-b588-c4e937cb02c2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:31.028813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 183/364] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/role_assignments?group.id=197daa5d6e3045e9beffe561914b2d9c&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:31.030091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0904b0-7e0f-4a80-a0a5-949f3bea3db6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.030387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0904b0-7e0f-4a80-a0a5-949f3bea3db6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.030684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0904b0-7e0f-4a80-a0a5-949f3bea3db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.054175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3b0904b0-7e0f-4a80-a0a5-949f3bea3db6 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:31.055558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3b0904b0-7e0f-4a80-a0a5-949f3bea3db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:31.104652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741bec6-b5ef-40d6-92d7-283ecb03f8f8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:31.145303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec15fd-0cb4-4274-aedb-877002ff0eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:31.146132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec15fd-0cb4-4274-aedb-877002ff0eac None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.146414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec15fd-0cb4-4274-aedb-877002ff0eac None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.146748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec15fd-0cb4-4274-aedb-877002ff0eac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.158462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec15fd-0cb4-4274-aedb-877002ff0eac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4dec15fd-0cb4-4274-aedb-877002ff0eac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:31.167195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:31.168757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:31.170109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 182/366] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:31.171013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 184/366] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:31.175905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec15fd-0cb4-4274-aedb-877002ff0eac None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:31.184342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0904b0-7e0f-4a80-a0a5-949f3bea3db6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:31.208229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:31.250230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:31.365427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec15fd-0cb4-4274-aedb-877002ff0eac None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:31.403237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdc890-0369-456f-a6dc-c46f406d296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:31.404487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdc890-0369-456f-a6dc-c46f406d296e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.404993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdc890-0369-456f-a6dc-c46f406d296e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.405513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdc890-0369-456f-a6dc-c46f406d296e None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.406964 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdc890-0369-456f-a6dc-c46f406d296e None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:31.408667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdc890-0369-456f-a6dc-c46f406d296e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:31.410933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 183/367] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:31 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:31.412025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdc890-0369-456f-a6dc-c46f406d296e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:31.432392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0904b0-7e0f-4a80-a0a5-949f3bea3db6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:31.478846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:31.479588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.479878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.480155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.480972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:31.481283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:31.482653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:31.491178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 185/368] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:31.495707 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:31.518624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:31.550617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdc890-0369-456f-a6dc-c46f406d296e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:31.593660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa361af-6c16-465e-89ca-729d05c208ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:31.595886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa361af-6c16-465e-89ca-729d05c208ba None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.596317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa361af-6c16-465e-89ca-729d05c208ba None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.596753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa361af-6c16-465e-89ca-729d05c208ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.598593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:31.613101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa361af-6c16-465e-89ca-729d05c208ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:31.615215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c1f21-f0ba-4b20-ac63-ee031405faa3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-baa361af-6c16-465e-89ca-729d05c208ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:31.622815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:31.624029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:31.625019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:31.627956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 184/369] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:05:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:31.630128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa361af-6c16-465e-89ca-729d05c208ba None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:31.667964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e21456-54ec-4821-93e2-ae249e10865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:31.669518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e21456-54ec-4821-93e2-ae249e10865e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.669857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e21456-54ec-4821-93e2-ae249e10865e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.670245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e21456-54ec-4821-93e2-ae249e10865e None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.671668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e21456-54ec-4821-93e2-ae249e10865e None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:31.673351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e21456-54ec-4821-93e2-ae249e10865e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:31.677917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 186/370] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:31 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:31.684745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e21456-54ec-4821-93e2-ae249e10865e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:31.758722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa361af-6c16-465e-89ca-729d05c208ba None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:31.814107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e21456-54ec-4821-93e2-ae249e10865e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:31.815058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497135e-396a-4ab8-bb4c-f19a3b458405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:31.815975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497135e-396a-4ab8-bb4c-f19a3b458405 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.816471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497135e-396a-4ab8-bb4c-f19a3b458405 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.816951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497135e-396a-4ab8-bb4c-f19a3b458405 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.821020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:31.860433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:31.861991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:31.862674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:31.863330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:31.864423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:31.864932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:31.866656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:31.878148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 187/371] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:31.879314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:31.909421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497135e-396a-4ab8-bb4c-f19a3b458405 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:31.910904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497135e-396a-4ab8-bb4c-f19a3b458405 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:31.911736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:32.017172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65143c7b-fe01-4c09-890b-2e35f21cdf9e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:32.034657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 185/372] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:32.039277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497135e-396a-4ab8-bb4c-f19a3b458405 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:32.071054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:32.071774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:32.072597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:32.072597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:32.073159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] RBAC: Adding query filter params (group.id=197daa5d6e3045e9beffe561914b2d9c, role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:32.073476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:32.075104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:32.090666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 188/373] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:31 2022] GET /identity/v3/role_assignments?group.id=197daa5d6e3045e9beffe561914b2d9c&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:32.090666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:32.115283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:32.181184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:32.194718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497135e-396a-4ab8-bb4c-f19a3b458405 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:32.203890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ade5a-e810-485c-8ec3-60823d0d027f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:32.234343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c546f-51e7-413e-8fe7-2fc776a67d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:32.235434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c546f-51e7-413e-8fe7-2fc776a67d8a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:32.235928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c546f-51e7-413e-8fe7-2fc776a67d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:32.236468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c546f-51e7-413e-8fe7-2fc776a67d8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:32.242919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5b001-d60a-4f08-be6d-f4ba5daaf9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:32.243831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5b001-d60a-4f08-be6d-f4ba5daaf9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:32.244120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5b001-d60a-4f08-be6d-f4ba5daaf9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:32.244401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5b001-d60a-4f08-be6d-f4ba5daaf9f9 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:32.245653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5b001-d60a-4f08-be6d-f4ba5daaf9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:32.247688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5b001-d60a-4f08-be6d-f4ba5daaf9f9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:32.249502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 189/374] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:32 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:32.254866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5b001-d60a-4f08-be6d-f4ba5daaf9f9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:32.317450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c546f-51e7-413e-8fe7-2fc776a67d8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:32.319343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c546f-51e7-413e-8fe7-2fc776a67d8a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:32.374903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:32.391189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5b001-d60a-4f08-be6d-f4ba5daaf9f9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:32.431939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861af20-bd4f-4252-85c6-336d72999a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:32.432725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861af20-bd4f-4252-85c6-336d72999a8e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:32.433009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861af20-bd4f-4252-85c6-336d72999a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:32.433284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861af20-bd4f-4252-85c6-336d72999a8e None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:32.435056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861af20-bd4f-4252-85c6-336d72999a8e None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:32.436627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861af20-bd4f-4252-85c6-336d72999a8e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:32.440158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 190/375] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:32 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:32.440390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 186/376] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:32.445925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861af20-bd4f-4252-85c6-336d72999a8e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:32.447465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c546f-51e7-413e-8fe7-2fc776a67d8a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:32.468158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:32.554840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c546f-51e7-413e-8fe7-2fc776a67d8a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:32.557013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861af20-bd4f-4252-85c6-336d72999a8e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:32.590930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:32.591957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:32.592473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:32.592954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:32.593990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] RBAC: Adding query filter params (group.id=197daa5d6e3045e9beffe561914b2d9c, role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:32.594938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:32.597765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:32.603610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:32.605464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 187/377] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:32 2022] GET /identity/v3/role_assignments?group.id=197daa5d6e3045e9beffe561914b2d9c&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:32.616132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293568e8-a61c-4f5e-88f4-23fc03bc1d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:32.617041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293568e8-a61c-4f5e-88f4-23fc03bc1d72 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:32.621920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293568e8-a61c-4f5e-88f4-23fc03bc1d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:32.621920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293568e8-a61c-4f5e-88f4-23fc03bc1d72 None admin] PATH_INFO: `/v3/projects/d16c61acfee74652b6cd4c256cd8f443` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:32.621920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293568e8-a61c-4f5e-88f4-23fc03bc1d72 None admin] RBAC: Authorizing `identity:get_project(project_id=d16c61acfee74652b6cd4c256cd8f443)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:32.621920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293568e8-a61c-4f5e-88f4-23fc03bc1d72 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:32.622577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 191/378] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:32 2022] GET /identity/v3/projects/d16c61acfee74652b6cd4c256cd8f443 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:32.628279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:32.771111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22987a3a-5e25-46cb-8a62-bb2d4f208adb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:32.813325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27aab1-9acd-4fa8-a36c-e59d1de50fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:32.814216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27aab1-9acd-4fa8-a36c-e59d1de50fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:32.814505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27aab1-9acd-4fa8-a36c-e59d1de50fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:32.814787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27aab1-9acd-4fa8-a36c-e59d1de50fb8 None admin] PATH_INFO: `/v3/groups/558475b058dd446f84d69c072ec0e144` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:32.820707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27aab1-9acd-4fa8-a36c-e59d1de50fb8 None admin] RBAC: Authorizing `identity:get_group(group_id=558475b058dd446f84d69c072ec0e144)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:32.822196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27aab1-9acd-4fa8-a36c-e59d1de50fb8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:32.824314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 188/379] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:32 2022] GET /identity/v3/groups/558475b058dd446f84d69c072ec0e144 => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:32.832448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293568e8-a61c-4f5e-88f4-23fc03bc1d72 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:32.936218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:32.952889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293568e8-a61c-4f5e-88f4-23fc03bc1d72 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:32.987656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399716f4-e2fc-43f4-a518-c357493110db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:32.988327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399716f4-e2fc-43f4-a518-c357493110db None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:32.988611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399716f4-e2fc-43f4-a518-c357493110db None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:32.988890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399716f4-e2fc-43f4-a518-c357493110db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:32.989730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399716f4-e2fc-43f4-a518-c357493110db None admin] RBAC: Adding query filter params (group.id=558475b058dd446f84d69c072ec0e144, role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=d16c61acfee74652b6cd4c256cd8f443) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:32.990045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399716f4-e2fc-43f4-a518-c357493110db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:32.991526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399716f4-e2fc-43f4-a518-c357493110db None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:32.998489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 192/380] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:32 2022] GET /identity/v3/role_assignments?group.id=558475b058dd446f84d69c072ec0e144&scope.project.id=d16c61acfee74652b6cd4c256cd8f443&role.id=007af6bd1b734517b934b4cac45faf2c => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:33.367898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de021c-f8c2-4537-aa8c-19fe21969205 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:33.368087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de021c-f8c2-4537-aa8c-19fe21969205 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:33.368372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de021c-f8c2-4537-aa8c-19fe21969205 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:33.370026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 189/381] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:33.389466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73c4d-f39a-4117-8a6a-9363fe7c6f2f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:33.389769 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73c4d-f39a-4117-8a6a-9363fe7c6f2f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:33.390058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73c4d-f39a-4117-8a6a-9363fe7c6f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:33.407624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-83a73c4d-f39a-4117-8a6a-9363fe7c6f2f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:33.409213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-83a73c4d-f39a-4117-8a6a-9363fe7c6f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:33.505740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 193/382] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:33.539109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de021c-f8c2-4537-aa8c-19fe21969205 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:33.558199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f0593-58df-4eaf-b620-b00ae5c10886 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:33.558485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f0593-58df-4eaf-b620-b00ae5c10886 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:33.558801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f0593-58df-4eaf-b620-b00ae5c10886 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:33.560305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 194/383] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:33.569374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da661-b362-4ea7-afa3-49b51a15c660 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:33.569661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da661-b362-4ea7-afa3-49b51a15c660 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:33.569942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da661-b362-4ea7-afa3-49b51a15c660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:33.596845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8a3da661-b362-4ea7-afa3-49b51a15c660 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:33.598052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8a3da661-b362-4ea7-afa3-49b51a15c660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:33.669186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:33.731906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 195/384] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 11 11:05:33.738743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de021c-f8c2-4537-aa8c-19fe21969205 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:33.774548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077304b5-c7f2-40d4-968e-cf89546159fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:33.775387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077304b5-c7f2-40d4-968e-cf89546159fa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:33.776253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077304b5-c7f2-40d4-968e-cf89546159fa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:33.776541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077304b5-c7f2-40d4-968e-cf89546159fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:33.785257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3da661-b362-4ea7-afa3-49b51a15c660 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:33.797001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:33.805402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077304b5-c7f2-40d4-968e-cf89546159fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-077304b5-c7f2-40d4-968e-cf89546159fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:33.814196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:33.815078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:33.815941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:33.819287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 190/385] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:33.824128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077304b5-c7f2-40d4-968e-cf89546159fa None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:33.961509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3da661-b362-4ea7-afa3-49b51a15c660 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:34.002953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077304b5-c7f2-40d4-968e-cf89546159fa None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:34.025072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a77fe-6038-41ba-b4a0-413e6d2492d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:34.025883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a77fe-6038-41ba-b4a0-413e6d2492d1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.026167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a77fe-6038-41ba-b4a0-413e6d2492d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.026445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a77fe-6038-41ba-b4a0-413e6d2492d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.041428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:34.046815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a77fe-6038-41ba-b4a0-413e6d2492d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-288a77fe-6038-41ba-b4a0-413e6d2492d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:34.056898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:34.060529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:34.061349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 196/386] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:34.068949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a77fe-6038-41ba-b4a0-413e6d2492d1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:34.073437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:34.073883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.074221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.075017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.075809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:34.076124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:34.078732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:34.096986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 191/387] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:34.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:34.214351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:34.269171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6842a69-05e6-4f76-a420-0984f5ae1596 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:34.273934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a77fe-6038-41ba-b4a0-413e6d2492d1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:34.319699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:34.320736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.321231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.321719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.322684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:34.323201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:34.325013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:34.332906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447acda9-51cd-4520-b279-bcd868301c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:34.333580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447acda9-51cd-4520-b279-bcd868301c66 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.333867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447acda9-51cd-4520-b279-bcd868301c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.334144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447acda9-51cd-4520-b279-bcd868301c66 None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.335397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447acda9-51cd-4520-b279-bcd868301c66 None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:34.336868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447acda9-51cd-4520-b279-bcd868301c66 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:34.337305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 197/388] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:34.338816 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 192/389] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:34 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:34.343226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:34.347829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447acda9-51cd-4520-b279-bcd868301c66 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:34.462555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:34.502219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943da74b-acd0-4389-8232-528b9880fdbc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:34.537062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:34.552832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d999b-b4ce-406c-b4af-907af389cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:34.553706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d999b-b4ce-406c-b4af-907af389cca8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.553905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447acda9-51cd-4520-b279-bcd868301c66 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:34.554008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d999b-b4ce-406c-b4af-907af389cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.554262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d999b-b4ce-406c-b4af-907af389cca8 None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.555511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d999b-b4ce-406c-b4af-907af389cca8 None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:34.556961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d999b-b4ce-406c-b4af-907af389cca8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:34.558832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 198/390] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:34 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:34.563896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d999b-b4ce-406c-b4af-907af389cca8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:34.597398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d6c9-f28a-4a85-b50d-64ed77e6282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:34.598625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d6c9-f28a-4a85-b50d-64ed77e6282c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.598913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d6c9-f28a-4a85-b50d-64ed77e6282c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.599186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4d6c9-f28a-4a85-b50d-64ed77e6282c None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.602874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4d6c9-f28a-4a85-b50d-64ed77e6282c None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:34.606798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4d6c9-f28a-4a85-b50d-64ed77e6282c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:34.608580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 193/391] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:34 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:34.614164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d6c9-f28a-4a85-b50d-64ed77e6282c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:34.744515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4d6c9-f28a-4a85-b50d-64ed77e6282c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:34.767007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:34.784865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642ad76e-7184-4a5e-88ce-a764b882a9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:34.786072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ad76e-7184-4a5e-88ce-a764b882a9c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.786515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ad76e-7184-4a5e-88ce-a764b882a9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.811220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642ad76e-7184-4a5e-88ce-a764b882a9c2 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755/groups/197daa5d6e3045e9beffe561914b2d9c/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.811220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ad76e-7184-4a5e-88ce-a764b882a9c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ca74ec2a86e48a28a003d344862f755, group_id=197daa5d6e3045e9beffe561914b2d9c, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:34.811454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d999b-b4ce-406c-b4af-907af389cca8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:34.811870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642ad76e-7184-4a5e-88ce-a764b882a9c2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:34.833281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dca84-a73d-4dbc-b708-933989c634fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:34.835527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dca84-a73d-4dbc-b708-933989c634fe None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.835956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dca84-a73d-4dbc-b708-933989c634fe None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.836376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dca84-a73d-4dbc-b708-933989c634fe None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.837701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dca84-a73d-4dbc-b708-933989c634fe None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:34.839687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dca84-a73d-4dbc-b708-933989c634fe None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:34.841744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 199/392] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:34 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:34.847394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dca84-a73d-4dbc-b708-933989c634fe None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:34.910135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 194/393] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 11 11:05:34 2022] PUT /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755/groups/197daa5d6e3045e9beffe561914b2d9c/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:34.918883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f448a1-0a37-418b-bae6-e60fe27add69 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.919457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f448a1-0a37-418b-bae6-e60fe27add69 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.919876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f448a1-0a37-418b-bae6-e60fe27add69 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.921864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 195/394] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:34.928485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc6ce3-18ac-497d-962d-e129a51730e3 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:34.929042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc6ce3-18ac-497d-962d-e129a51730e3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:34.929770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc6ce3-18ac-497d-962d-e129a51730e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:34.941461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:34.968937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b3fc6ce3-18ac-497d-962d-e129a51730e3 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:34.972548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b3fc6ce3-18ac-497d-962d-e129a51730e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:35.021002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dca84-a73d-4dbc-b708-933989c634fe None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:35.095522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca53a7-6f09-4366-8f68-fc1797ac908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:35.095522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca53a7-6f09-4366-8f68-fc1797ac908d None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.095789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca53a7-6f09-4366-8f68-fc1797ac908d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.096027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca53a7-6f09-4366-8f68-fc1797ac908d None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755/groups/197daa5d6e3045e9beffe561914b2d9c/roles/9eaabfe45189403f8cee9512fefbba65` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.100204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca53a7-6f09-4366-8f68-fc1797ac908d None admin] RBAC: Authorizing `identity:create_grant(project_id=6ca74ec2a86e48a28a003d344862f755, group_id=197daa5d6e3045e9beffe561914b2d9c, role_id=9eaabfe45189403f8cee9512fefbba65)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:35.101665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca53a7-6f09-4366-8f68-fc1797ac908d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:35.105939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 196/395] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:35.128007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc6ce3-18ac-497d-962d-e129a51730e3 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:35.185483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 200/396] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 11 11:05:34 2022] PUT /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755/groups/197daa5d6e3045e9beffe561914b2d9c/roles/9eaabfe45189403f8cee9512fefbba65 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:05:35.247419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc6ce3-18ac-497d-962d-e129a51730e3 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:35.259482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:35.285672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47991c-c04b-48dd-b074-777d721db1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:35.286394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47991c-c04b-48dd-b074-777d721db1fc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.286679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47991c-c04b-48dd-b074-777d721db1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.286958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47991c-c04b-48dd-b074-777d721db1fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.299656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47991c-c04b-48dd-b074-777d721db1fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1a47991c-c04b-48dd-b074-777d721db1fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:35.309015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:35.309974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:35.310805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:35.315186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 197/397] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:35.316589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca53a7-6f09-4366-8f68-fc1797ac908d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:35.435120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388be1-7711-4b2d-aac5-1e38f9cd4fff None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.435477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388be1-7711-4b2d-aac5-1e38f9cd4fff None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.435761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388be1-7711-4b2d-aac5-1e38f9cd4fff None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.437147 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 198/398] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:35.442600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e03a-3ebb-479d-b8c8-1f403c735f84 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.443129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e03a-3ebb-479d-b8c8-1f403c735f84 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.443671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644e03a-3ebb-479d-b8c8-1f403c735f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.465601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3644e03a-3ebb-479d-b8c8-1f403c735f84 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:35.466736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca53a7-6f09-4366-8f68-fc1797ac908d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:35.467546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3644e03a-3ebb-479d-b8c8-1f403c735f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:35.494147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:35.507453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:35.509411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.509411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.509411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.510240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:35.510748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:35.513068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:35.522864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 201/399] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:35.528888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:35.557607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 199/400] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:35.559535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad2a63-e323-4f10-b319-1efe76fe0486 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.559971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad2a63-e323-4f10-b319-1efe76fe0486 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.560418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad2a63-e323-4f10-b319-1efe76fe0486 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.562100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 200/401] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:35.587805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f93c7-cb20-47f0-b3f8-dcfa383daa03 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.588423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f93c7-cb20-47f0-b3f8-dcfa383daa03 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.588954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f93c7-cb20-47f0-b3f8-dcfa383daa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.609090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1a8f93c7-cb20-47f0-b3f8-dcfa383daa03 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:35.610594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1a8f93c7-cb20-47f0-b3f8-dcfa383daa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:35.650642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b14904-4034-4e75-98b0-b2554c82b14c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:35.687231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5bfa9-b6d0-4613-8691-2c699946811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:35.688158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5bfa9-b6d0-4613-8691-2c699946811f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.688440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5bfa9-b6d0-4613-8691-2c699946811f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.688717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5bfa9-b6d0-4613-8691-2c699946811f None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.689969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5bfa9-b6d0-4613-8691-2c699946811f None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:35.691490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5bfa9-b6d0-4613-8691-2c699946811f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:35.697621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 202/402] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:35 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:35.698576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5bfa9-b6d0-4613-8691-2c699946811f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:35.703769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 201/403] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:35.704863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f93c7-cb20-47f0-b3f8-dcfa383daa03 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:35.739972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:35.782906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:35.834526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5bfa9-b6d0-4613-8691-2c699946811f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:35.846343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f93c7-cb20-47f0-b3f8-dcfa383daa03 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:35.872568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5452aec-68c1-4aea-bda0-9e666670e1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:35.873407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5452aec-68c1-4aea-bda0-9e666670e1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.873690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5452aec-68c1-4aea-bda0-9e666670e1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.873968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5452aec-68c1-4aea-bda0-9e666670e1f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.885064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5452aec-68c1-4aea-bda0-9e666670e1f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:35.892216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ea4e-83d0-482e-9dd4-a3f3a907dd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:35.893305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ea4e-83d0-482e-9dd4-a3f3a907dd72 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e5452aec-68c1-4aea-bda0-9e666670e1f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:35.893410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:35.894360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:35.895175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 203/404] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:35.896704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ea4e-83d0-482e-9dd4-a3f3a907dd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:35.897179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63ea4e-83d0-482e-9dd4-a3f3a907dd72 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:35.898643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63ea4e-83d0-482e-9dd4-a3f3a907dd72 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:35.900348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63ea4e-83d0-482e-9dd4-a3f3a907dd72 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:35.900454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5452aec-68c1-4aea-bda0-9e666670e1f0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:35.903382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 202/405] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:35 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:35.904381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ea4e-83d0-482e-9dd4-a3f3a907dd72 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:36.011989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5452aec-68c1-4aea-bda0-9e666670e1f0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.030670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:36.035218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63ea4e-83d0-482e-9dd4-a3f3a907dd72 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.037338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:36.066766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760737fd-6983-420f-b015-1db7038f18b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.068111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760737fd-6983-420f-b015-1db7038f18b2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.068903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760737fd-6983-420f-b015-1db7038f18b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.069536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760737fd-6983-420f-b015-1db7038f18b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.076222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.077052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.077716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.078022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.078933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:36.079466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:36.080965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:36.085670 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760737fd-6983-420f-b015-1db7038f18b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-760737fd-6983-420f-b015-1db7038f18b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:36.095344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:36.096401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:36.097214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:36.097833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 203/406] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:36.097833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:36.099758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 204/407] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:05:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:36.100746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760737fd-6983-420f-b015-1db7038f18b2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:36.212488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718a50a-ed6b-437e-a4d3-cdd28f4bb094 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.236656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760737fd-6983-420f-b015-1db7038f18b2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.259034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7584-fa73-40f7-bc76-03016f4a26ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.260311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7584-fa73-40f7-bc76-03016f4a26ba None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.260602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7584-fa73-40f7-bc76-03016f4a26ba None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.260877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b7584-fa73-40f7-bc76-03016f4a26ba None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/groups/197daa5d6e3045e9beffe561914b2d9c/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.263409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b7584-fa73-40f7-bc76-03016f4a26ba None admin] RBAC: Authorizing `identity:create_grant(project_id=bf653a9fe09a4a3e86a6e3e8c134711e, group_id=197daa5d6e3045e9beffe561914b2d9c, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:36.265519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b7584-fa73-40f7-bc76-03016f4a26ba None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:36.279158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a3a1b-361c-4751-a8c1-5c793b157f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.281773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a3a1b-361c-4751-a8c1-5c793b157f1a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.281773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a3a1b-361c-4751-a8c1-5c793b157f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.281773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a3a1b-361c-4751-a8c1-5c793b157f1a None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.281972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a3a1b-361c-4751-a8c1-5c793b157f1a None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:36.283327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a3a1b-361c-4751-a8c1-5c793b157f1a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:36.284929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 205/408] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:36 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:36.289466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a3a1b-361c-4751-a8c1-5c793b157f1a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:36.336415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:36.361928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 204/409] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 11 11:05:36 2022] PUT /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/groups/197daa5d6e3045e9beffe561914b2d9c/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:36.361928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7584-fa73-40f7-bc76-03016f4a26ba None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:36.414581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:36.438085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a3a1b-361c-4751-a8c1-5c793b157f1a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.515819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.516761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.516761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.519278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.519278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:36.519278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:36.519576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:36.538218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 206/410] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:36.544178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:36.577550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b7584-fa73-40f7-bc76-03016f4a26ba None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.616497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d5a7e-f34b-48a0-9a4e-0db26c49cb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.617305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d5a7e-f34b-48a0-9a4e-0db26c49cb41 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.617659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d5a7e-f34b-48a0-9a4e-0db26c49cb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.618001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d5a7e-f34b-48a0-9a4e-0db26c49cb41 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.619689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d5a7e-f34b-48a0-9a4e-0db26c49cb41 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:36.620913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d5a7e-f34b-48a0-9a4e-0db26c49cb41 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:36.623392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 205/411] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:36 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:36.629269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d5a7e-f34b-48a0-9a4e-0db26c49cb41 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:36.674338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc49f-0b4e-46db-850c-27b82249fbd3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.683050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:36.713286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7b96e-610d-4c93-9a1b-cc65ba47a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.714515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7b96e-610d-4c93-9a1b-cc65ba47a66e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.715547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7b96e-610d-4c93-9a1b-cc65ba47a66e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.716003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7b96e-610d-4c93-9a1b-cc65ba47a66e None admin] PATH_INFO: `/v3/groups/558475b058dd446f84d69c072ec0e144` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.723651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7b96e-610d-4c93-9a1b-cc65ba47a66e None admin] RBAC: Authorizing `identity:get_group(group_id=558475b058dd446f84d69c072ec0e144)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:36.725548 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7b96e-610d-4c93-9a1b-cc65ba47a66e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:36.727718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 207/412] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:36 2022] GET /identity/v3/groups/558475b058dd446f84d69c072ec0e144 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:36.732598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:36.734128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7b96e-610d-4c93-9a1b-cc65ba47a66e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:36.742689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d5a7e-f34b-48a0-9a4e-0db26c49cb41 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.779841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc1c1f-a352-4193-a721-c877526949cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.780967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1c1f-a352-4193-a721-c877526949cd None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.781317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1c1f-a352-4193-a721-c877526949cd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.781656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc1c1f-a352-4193-a721-c877526949cd None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/groups/197daa5d6e3045e9beffe561914b2d9c/roles/9eaabfe45189403f8cee9512fefbba65` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.784103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc1c1f-a352-4193-a721-c877526949cd None admin] RBAC: Authorizing `identity:create_grant(project_id=bf653a9fe09a4a3e86a6e3e8c134711e, group_id=197daa5d6e3045e9beffe561914b2d9c, role_id=9eaabfe45189403f8cee9512fefbba65)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:36.785767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc1c1f-a352-4193-a721-c877526949cd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:36.872738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7b96e-610d-4c93-9a1b-cc65ba47a66e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:36.876732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 206/413] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 11 11:05:36 2022] PUT /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e/groups/197daa5d6e3045e9beffe561914b2d9c/roles/9eaabfe45189403f8cee9512fefbba65 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:05:36.912302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1a5ca-9fae-4f01-b62f-29c7a0265bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:36.913567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a5ca-9fae-4f01-b62f-29c7a0265bef None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:36.914035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a5ca-9fae-4f01-b62f-29c7a0265bef None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:36.914494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1a5ca-9fae-4f01-b62f-29c7a0265bef None admin] PATH_INFO: `/v3/projects/d16c61acfee74652b6cd4c256cd8f443` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:36.916068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1a5ca-9fae-4f01-b62f-29c7a0265bef None admin] RBAC: Authorizing `identity:get_project(project_id=d16c61acfee74652b6cd4c256cd8f443)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:36.917761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1a5ca-9fae-4f01-b62f-29c7a0265bef None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:36.920752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 208/414] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:36 2022] GET /identity/v3/projects/d16c61acfee74652b6cd4c256cd8f443 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:36.926693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc1c1f-a352-4193-a721-c877526949cd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:37.070412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc1c1f-a352-4193-a721-c877526949cd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:37.110390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a7325-11e5-4e20-815f-369311e2c150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:37.111312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7325-11e5-4e20-815f-369311e2c150 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:37.111603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7325-11e5-4e20-815f-369311e2c150 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:37.111884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7325-11e5-4e20-815f-369311e2c150 None admin] PATH_INFO: `/v3/projects/d16c61acfee74652b6cd4c256cd8f443/groups/558475b058dd446f84d69c072ec0e144/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:37.115768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a7325-11e5-4e20-815f-369311e2c150 None admin] RBAC: Authorizing `identity:create_grant(project_id=d16c61acfee74652b6cd4c256cd8f443, group_id=558475b058dd446f84d69c072ec0e144, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:37.117242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a7325-11e5-4e20-815f-369311e2c150 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:37.173019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291943-efb9-47ce-b6d9-0c285f444c43 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:37.173310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291943-efb9-47ce-b6d9-0c285f444c43 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:37.173640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d291943-efb9-47ce-b6d9-0c285f444c43 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:37.175596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 209/415] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:05:37.180493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7ba8-3b6c-480f-89a9-39dfee053944 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:37.180783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7ba8-3b6c-480f-89a9-39dfee053944 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:37.181067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac7ba8-3b6c-480f-89a9-39dfee053944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:37.194013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 207/416] 10.222.0.26 () {58 vars in 1442 bytes} [Mon Jul 11 11:05:36 2022] PUT /identity/v3/projects/d16c61acfee74652b6cd4c256cd8f443/groups/558475b058dd446f84d69c072ec0e144/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:37.203969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-15ac7ba8-3b6c-480f-89a9-39dfee053944 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:37.205244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-15ac7ba8-3b6c-480f-89a9-39dfee053944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:37.256418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:37.300172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 210/417] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:37.327555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a7325-11e5-4e20-815f-369311e2c150 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:37.340880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:37.457540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a7325-11e5-4e20-815f-369311e2c150 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:37.499972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e37bd-938a-4b7f-b8dc-35e49a6a0b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:37.500775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e37bd-938a-4b7f-b8dc-35e49a6a0b87 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:37.501063 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e37bd-938a-4b7f-b8dc-35e49a6a0b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:37.501342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e37bd-938a-4b7f-b8dc-35e49a6a0b87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:37.514241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e37bd-938a-4b7f-b8dc-35e49a6a0b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-6b2e37bd-938a-4b7f-b8dc-35e49a6a0b87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:37.522076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:37.522970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:37.524417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 208/418] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:37.529610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac7ba8-3b6c-480f-89a9-39dfee053944 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:37.571727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99890fee-5720-4fb4-a4e1-0a349c375fa4 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:37.572049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99890fee-5720-4fb4-a4e1-0a349c375fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:37.572296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99890fee-5720-4fb4-a4e1-0a349c375fa4 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:37.573971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 209/419] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:37.580283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bffad-bd53-459c-99d4-d32cd84147e9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:37.580575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bffad-bd53-459c-99d4-d32cd84147e9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:37.580859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bffad-bd53-459c-99d4-d32cd84147e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:37.613232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f46bffad-bd53-459c-99d4-d32cd84147e9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:37.616212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f46bffad-bd53-459c-99d4-d32cd84147e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:37.685516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac7ba8-3b6c-480f-89a9-39dfee053944 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:37.737571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:37.739757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:37.743538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:37.743538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:37.743538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:37.743538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:37.743538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:37.763519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 211/420] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:37.766649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:37.769205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:37.776669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 210/421] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:37.815312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:37.816486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bffad-bd53-459c-99d4-d32cd84147e9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:37.931926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0877f-f224-4f9c-9171-6ebdf2da7b1a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:37.972505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd085-5c2c-44e5-ad34-43305414998a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:37.973172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd085-5c2c-44e5-ad34-43305414998a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:37.973459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd085-5c2c-44e5-ad34-43305414998a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:37.973738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afd085-5c2c-44e5-ad34-43305414998a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:38.020817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bffad-bd53-459c-99d4-d32cd84147e9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:38.093621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afd085-5c2c-44e5-ad34-43305414998a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:38.095113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afd085-5c2c-44e5-ad34-43305414998a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:38.097483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e319c46-ad16-417f-914f-bf53b18e5818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:38.098302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e319c46-ad16-417f-914f-bf53b18e5818 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:38.098584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e319c46-ad16-417f-914f-bf53b18e5818 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:38.098861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e319c46-ad16-417f-914f-bf53b18e5818 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:38.119879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e319c46-ad16-417f-914f-bf53b18e5818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-3e319c46-ad16-417f-914f-bf53b18e5818 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:38.131278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:38.132208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:38.133029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:38.140992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 211/422] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:38.142520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e319c46-ad16-417f-914f-bf53b18e5818 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:38.163640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:38.266401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:38.281777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 212/423] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:38.302853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd085-5c2c-44e5-ad34-43305414998a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:38.341750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e319c46-ad16-417f-914f-bf53b18e5818 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:38.378619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:38.379314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:38.379601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:38.379881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:38.380646 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:38.380956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:38.382331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:38.400425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 212/424] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:38.405819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:38.528203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afd085-5c2c-44e5-ad34-43305414998a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:38.594302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb74ad7-83b1-4f92-a4ef-898fc3813331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:38.595037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb74ad7-83b1-4f92-a4ef-898fc3813331 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:38.595345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb74ad7-83b1-4f92-a4ef-898fc3813331 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:38.595794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb74ad7-83b1-4f92-a4ef-898fc3813331 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:38.597635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb74ad7-83b1-4f92-a4ef-898fc3813331 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:38.600002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb74ad7-83b1-4f92-a4ef-898fc3813331 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:38.602140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 213/425] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:38 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:38.609593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb74ad7-83b1-4f92-a4ef-898fc3813331 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:38.613329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:38.632555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdd4ac-a5cf-4e93-bbd7-381ada4d3238 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:38.672665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:38.722832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c571f0b-94ab-4d83-bacf-b11b753f88f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:38.723530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571f0b-94ab-4d83-bacf-b11b753f88f8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:38.723819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571f0b-94ab-4d83-bacf-b11b753f88f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:38.724101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c571f0b-94ab-4d83-bacf-b11b753f88f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:38.764268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb74ad7-83b1-4f92-a4ef-898fc3813331 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:38.823130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58823c-3d4f-4f1c-b272-6c1e08b2f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:38.825337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58823c-3d4f-4f1c-b272-6c1e08b2f9bd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:38.826782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58823c-3d4f-4f1c-b272-6c1e08b2f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:38.827281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58823c-3d4f-4f1c-b272-6c1e08b2f9bd None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:38.832588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58823c-3d4f-4f1c-b272-6c1e08b2f9bd None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:38.834272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58823c-3d4f-4f1c-b272-6c1e08b2f9bd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:38.843629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 214/426] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:38 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:38.858343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58823c-3d4f-4f1c-b272-6c1e08b2f9bd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:38.870797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c571f0b-94ab-4d83-bacf-b11b753f88f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:38.873830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c571f0b-94ab-4d83-bacf-b11b753f88f8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:38.991623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:39.024231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58823c-3d4f-4f1c-b272-6c1e08b2f9bd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:39.038527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 213/427] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:39.044816 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c2c54-ead7-43a2-9bf0-d7875a6c506c None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.045433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c2c54-ead7-43a2-9bf0-d7875a6c506c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.045932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c2c54-ead7-43a2-9bf0-d7875a6c506c None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.047484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 214/428] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:39.048442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c2c54-ead7-43a2-9bf0-d7875a6c506c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:39.071398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:39.076152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5775d-eabd-46eb-8340-c136cbd66ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:39.076823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5775d-eabd-46eb-8340-c136cbd66ead None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.077317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5775d-eabd-46eb-8340-c136cbd66ead None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.077423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5775d-eabd-46eb-8340-c136cbd66ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.078212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5775d-eabd-46eb-8340-c136cbd66ead None admin] RBAC: Adding query filter params (group.id=197daa5d6e3045e9beffe561914b2d9c, role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=6ca74ec2a86e48a28a003d344862f755) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:39.078521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5775d-eabd-46eb-8340-c136cbd66ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:39.080049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5775d-eabd-46eb-8340-c136cbd66ead None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:39.091278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 215/429] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:38 2022] GET /identity/v3/role_assignments?group.id=197daa5d6e3045e9beffe561914b2d9c&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:39.091278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec110e7-70c7-4672-b9dc-e9ef800e7abe None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.091278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec110e7-70c7-4672-b9dc-e9ef800e7abe None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.091278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec110e7-70c7-4672-b9dc-e9ef800e7abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.120181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cec110e7-70c7-4672-b9dc-e9ef800e7abe None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:39.121900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cec110e7-70c7-4672-b9dc-e9ef800e7abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:39.242480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 216/430] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:39.248659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c2c54-ead7-43a2-9bf0-d7875a6c506c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:39.266567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec110e7-70c7-4672-b9dc-e9ef800e7abe None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:39.301104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aeec1-29f2-45cc-b4cf-dc48f4945107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:39.302382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aeec1-29f2-45cc-b4cf-dc48f4945107 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.302859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aeec1-29f2-45cc-b4cf-dc48f4945107 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.303373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aeec1-29f2-45cc-b4cf-dc48f4945107 None admin] PATH_INFO: `/v3/projects/6ca74ec2a86e48a28a003d344862f755` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.304861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59aeec1-29f2-45cc-b4cf-dc48f4945107 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca74ec2a86e48a28a003d344862f755)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:39.306535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59aeec1-29f2-45cc-b4cf-dc48f4945107 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:39.309163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 215/431] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:39 2022] GET /identity/v3/projects/6ca74ec2a86e48a28a003d344862f755 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:39.310742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e0cb3-fd9b-4069-8de2-428c3304a591 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.311134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e0cb3-fd9b-4069-8de2-428c3304a591 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.311440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e0cb3-fd9b-4069-8de2-428c3304a591 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.312876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 216/432] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:39.318597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f4d7d-eaaa-418f-8ff8-d900a7092e2d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.318886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f4d7d-eaaa-418f-8ff8-d900a7092e2d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.319156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f4d7d-eaaa-418f-8ff8-d900a7092e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.335709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:39.341133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5d4f4d7d-eaaa-418f-8ff8-d900a7092e2d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:39.343204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5d4f4d7d-eaaa-418f-8ff8-d900a7092e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:39.383192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec110e7-70c7-4672-b9dc-e9ef800e7abe None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:39.424160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aea4c4-52cb-401d-99b2-26a4badaf150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:39.425515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aea4c4-52cb-401d-99b2-26a4badaf150 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.425973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aea4c4-52cb-401d-99b2-26a4badaf150 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.426436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aea4c4-52cb-401d-99b2-26a4badaf150 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.432031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 217/433] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:39.433045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f4d7d-eaaa-418f-8ff8-d900a7092e2d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:39.453069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aea4c4-52cb-401d-99b2-26a4badaf150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:39.456485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-58aea4c4-52cb-401d-99b2-26a4badaf150 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:39.465107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:39.472408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 217/434] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:39.473234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aea4c4-52cb-401d-99b2-26a4badaf150 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:39.561196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f4d7d-eaaa-418f-8ff8-d900a7092e2d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:39.599862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aea4c4-52cb-401d-99b2-26a4badaf150 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:39.606548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad56c2-e371-40a4-b6c9-b0dcab0a48b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:39.607492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad56c2-e371-40a4-b6c9-b0dcab0a48b6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.607778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad56c2-e371-40a4-b6c9-b0dcab0a48b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.608489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad56c2-e371-40a4-b6c9-b0dcab0a48b6 None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.609701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad56c2-e371-40a4-b6c9-b0dcab0a48b6 None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:39.611898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad56c2-e371-40a4-b6c9-b0dcab0a48b6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:39.613985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 218/435] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:39 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:39.614641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad56c2-e371-40a4-b6c9-b0dcab0a48b6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:39.626445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:39.640155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9145-b16e-4513-bb3f-4742c567dd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:39.640155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9145-b16e-4513-bb3f-4742c567dd2a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.640155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9145-b16e-4513-bb3f-4742c567dd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.640155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9145-b16e-4513-bb3f-4742c567dd2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.650626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a9145-b16e-4513-bb3f-4742c567dd2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-790a9145-b16e-4513-bb3f-4742c567dd2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:39.659450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:39.660403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:39.661211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 218/436] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:39.662466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9145-b16e-4513-bb3f-4742c567dd2a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:39.703120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:39.717633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad56c2-e371-40a4-b6c9-b0dcab0a48b6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:39.750840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdc9b6-3670-429d-91fa-8b9c48a1b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:39.751573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc9b6-3670-429d-91fa-8b9c48a1b863 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.767470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc9b6-3670-429d-91fa-8b9c48a1b863 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.767470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc9b6-3670-429d-91fa-8b9c48a1b863 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.767470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc9b6-3670-429d-91fa-8b9c48a1b863 None admin] RBAC: Adding query filter params (name=member) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:39.767470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc9b6-3670-429d-91fa-8b9c48a1b863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:39.767470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc9b6-3670-429d-91fa-8b9c48a1b863 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:39.767806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9145-b16e-4513-bb3f-4742c567dd2a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:39.767897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 219/437] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:05:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:39.767897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2000ae-a92a-4ce6-9edd-0a5bc4081175 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.767897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2000ae-a92a-4ce6-9edd-0a5bc4081175 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.767897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2000ae-a92a-4ce6-9edd-0a5bc4081175 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.767897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 220/438] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:39.767897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2000ae-a92a-4ce6-9edd-0a5bc4081175 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:39.797312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:39.798010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.798294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.798569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.799435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] RBAC: Adding query filter params (group.id=197daa5d6e3045e9beffe561914b2d9c, role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=6ca74ec2a86e48a28a003d344862f755) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:39.799752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:39.801446 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:39.848197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 219/439] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:39 2022] GET /identity/v3/role_assignments?group.id=197daa5d6e3045e9beffe561914b2d9c&scope.project.id=6ca74ec2a86e48a28a003d344862f755&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:39.849108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:39.872011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2000ae-a92a-4ce6-9edd-0a5bc4081175 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:39.903322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:39.914617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fba165-f6c6-4d7b-be31-fbbee83b83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:39.915333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fba165-f6c6-4d7b-be31-fbbee83b83ea None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.915621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fba165-f6c6-4d7b-be31-fbbee83b83ea None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.917957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fba165-f6c6-4d7b-be31-fbbee83b83ea None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.917957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fba165-f6c6-4d7b-be31-fbbee83b83ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:39.917957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fba165-f6c6-4d7b-be31-fbbee83b83ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:39.918635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fba165-f6c6-4d7b-be31-fbbee83b83ea None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:39.932757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 221/440] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:39.932757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8921b1-4a9c-48ba-9ec0-46c46e41754b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:39.932757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8921b1-4a9c-48ba-9ec0-46c46e41754b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:39.932757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8921b1-4a9c-48ba-9ec0-46c46e41754b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:39.950222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-be8921b1-4a9c-48ba-9ec0-46c46e41754b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:39.951800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-be8921b1-4a9c-48ba-9ec0-46c46e41754b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:39.957116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:39.961559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8043-9284-4f6f-9f86-f3baef91d8db None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.000572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac1cb2-01a5-4602-b82a-236a591bc471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.000572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac1cb2-01a5-4602-b82a-236a591bc471 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.002096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac1cb2-01a5-4602-b82a-236a591bc471 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.002096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac1cb2-01a5-4602-b82a-236a591bc471 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.036504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 222/441] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:40.037443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8921b1-4a9c-48ba-9ec0-46c46e41754b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.092442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac1cb2-01a5-4602-b82a-236a591bc471 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.093908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac1cb2-01a5-4602-b82a-236a591bc471 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.153537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:40.162385 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8921b1-4a9c-48ba-9ec0-46c46e41754b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.200761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67dcad-04a1-4d28-9c30-c46fe1baf7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.201661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67dcad-04a1-4d28-9c30-c46fe1baf7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.201941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67dcad-04a1-4d28-9c30-c46fe1baf7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.202218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67dcad-04a1-4d28-9c30-c46fe1baf7b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.214519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:40.221581 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 220/442] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:40.221581 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac1cb2-01a5-4602-b82a-236a591bc471 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.273574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67dcad-04a1-4d28-9c30-c46fe1baf7b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.275140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67dcad-04a1-4d28-9c30-c46fe1baf7b4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.320720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac1cb2-01a5-4602-b82a-236a591bc471 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.353012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046f35e-1fe6-4bb8-9dac-ed42f795d984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.353796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046f35e-1fe6-4bb8-9dac-ed42f795d984 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.354073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046f35e-1fe6-4bb8-9dac-ed42f795d984 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.354345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046f35e-1fe6-4bb8-9dac-ed42f795d984 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.364254 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046f35e-1fe6-4bb8-9dac-ed42f795d984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:40.370338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 223/443] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:40.371903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67dcad-04a1-4d28-9c30-c46fe1baf7b4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2046f35e-1fe6-4bb8-9dac-ed42f795d984 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:40.372376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:40.373264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:40.374112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 221/444] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:40.376632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046f35e-1fe6-4bb8-9dac-ed42f795d984 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.388672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:40.414123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:40.471081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67dcad-04a1-4d28-9c30-c46fe1baf7b4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.476518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046f35e-1fe6-4bb8-9dac-ed42f795d984 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.504705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784fe51-89f1-4535-8109-915b5031b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.505558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784fe51-89f1-4535-8109-915b5031b216 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.505840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784fe51-89f1-4535-8109-915b5031b216 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.506114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784fe51-89f1-4535-8109-915b5031b216 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.507305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784fe51-89f1-4535-8109-915b5031b216 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.508733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784fe51-89f1-4535-8109-915b5031b216 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.509039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84401a-8652-4eeb-a208-29bdcf98a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.509896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84401a-8652-4eeb-a208-29bdcf98a205 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.510183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84401a-8652-4eeb-a208-29bdcf98a205 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.510420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 224/445] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:40.510514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84401a-8652-4eeb-a208-29bdcf98a205 None admin] PATH_INFO: `/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.511179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784fe51-89f1-4535-8109-915b5031b216 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.511701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84401a-8652-4eeb-a208-29bdcf98a205 None admin] RBAC: Authorizing `identity:get_project(project_id=bf653a9fe09a4a3e86a6e3e8c134711e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.513139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84401a-8652-4eeb-a208-29bdcf98a205 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.515038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 222/446] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/projects/bf653a9fe09a4a3e86a6e3e8c134711e => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:40.515799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84401a-8652-4eeb-a208-29bdcf98a205 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.589173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:40.611585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84401a-8652-4eeb-a208-29bdcf98a205 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.613060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784fe51-89f1-4535-8109-915b5031b216 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.622312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:40.645215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cbbe7-292b-4f82-afa4-50be081d5c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.646063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cbbe7-292b-4f82-afa4-50be081d5c7e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.646165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.646343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cbbe7-292b-4f82-afa4-50be081d5c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.646616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990cbbe7-292b-4f82-afa4-50be081d5c7e None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.646805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.647003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.647312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.647842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cbbe7-292b-4f82-afa4-50be081d5c7e None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.648065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:40.648377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.649262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990cbbe7-292b-4f82-afa4-50be081d5c7e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.649722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.651178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 223/447] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:40.651661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cbbe7-292b-4f82-afa4-50be081d5c7e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.659014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 225/448] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:40.660114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.752347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990cbbe7-292b-4f82-afa4-50be081d5c7e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.756243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1d898-3fdd-44bc-bd69-e254d02af55a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.783183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3811736-342a-45b2-8caf-b0862d1d25ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.788118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3811736-342a-45b2-8caf-b0862d1d25ec None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.788118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3811736-342a-45b2-8caf-b0862d1d25ec None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.788118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3811736-342a-45b2-8caf-b0862d1d25ec None admin] PATH_INFO: `/v3/groups/197daa5d6e3045e9beffe561914b2d9c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.788118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3811736-342a-45b2-8caf-b0862d1d25ec None admin] RBAC: Authorizing `identity:get_group(group_id=197daa5d6e3045e9beffe561914b2d9c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.788118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3811736-342a-45b2-8caf-b0862d1d25ec None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.788814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 224/449] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/groups/197daa5d6e3045e9beffe561914b2d9c => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:40.794440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.794546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3811736-342a-45b2-8caf-b0862d1d25ec None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.794629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.794629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.794629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.794629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] RBAC: Adding query filter params (group.id=197daa5d6e3045e9beffe561914b2d9c, role.id=9d899c1b7b0740348cc8f312ed536187, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:40.794629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.794629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.799637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 226/450] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/role_assignments?group.id=197daa5d6e3045e9beffe561914b2d9c&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=9d899c1b7b0740348cc8f312ed536187 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:40.800661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:40.811150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:40.844534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:40.908479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3811736-342a-45b2-8caf-b0862d1d25ec None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.919045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f2a1a-81fd-4424-8e23-9a385bf00ec1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:40.946578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8670b3-b123-423c-a61c-9ac94a0ce10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.947239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8670b3-b123-423c-a61c-9ac94a0ce10e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.947570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8670b3-b123-423c-a61c-9ac94a0ce10e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.947841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8670b3-b123-423c-a61c-9ac94a0ce10e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.960447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:40.961137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:40.961423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:40.961700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:40.962542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] RBAC: Adding query filter params (group.id=197daa5d6e3045e9beffe561914b2d9c, role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:40.962857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:40.964384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:40.971763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 227/451] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/role_assignments?group.id=197daa5d6e3045e9beffe561914b2d9c&scope.project.id=bf653a9fe09a4a3e86a6e3e8c134711e&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:41.030945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8670b3-b123-423c-a61c-9ac94a0ce10e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:41.032385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8670b3-b123-423c-a61c-9ac94a0ce10e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:41.050024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:41.152699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 225/452] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:41.158553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:41.258733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:41.263970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78631e7c-454e-499e-a8f1-42839554bcff None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:41.298505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8e394-ca73-44a6-9e56-9e2d3382830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:41.299329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8e394-ca73-44a6-9e56-9e2d3382830d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:41.299603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8e394-ca73-44a6-9e56-9e2d3382830d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:41.300134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8e394-ca73-44a6-9e56-9e2d3382830d None admin] PATH_INFO: `/v3/projects/d16c61acfee74652b6cd4c256cd8f443` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:41.301323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8e394-ca73-44a6-9e56-9e2d3382830d None admin] RBAC: Authorizing `identity:get_project(project_id=d16c61acfee74652b6cd4c256cd8f443)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:41.302770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8e394-ca73-44a6-9e56-9e2d3382830d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:41.304501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 228/453] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:05:41 2022] GET /identity/v3/projects/d16c61acfee74652b6cd4c256cd8f443 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:41.310336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8670b3-b123-423c-a61c-9ac94a0ce10e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:41.407578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8670b3-b123-423c-a61c-9ac94a0ce10e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:41.420093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:41.439994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef15d7-757e-4609-86a0-75d5b1b7259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:41.440842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef15d7-757e-4609-86a0-75d5b1b7259e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:41.441123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef15d7-757e-4609-86a0-75d5b1b7259e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:41.441395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef15d7-757e-4609-86a0-75d5b1b7259e None admin] PATH_INFO: `/v3/groups/558475b058dd446f84d69c072ec0e144` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:41.442600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef15d7-757e-4609-86a0-75d5b1b7259e None admin] RBAC: Authorizing `identity:get_group(group_id=558475b058dd446f84d69c072ec0e144)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:41.444380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef15d7-757e-4609-86a0-75d5b1b7259e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:41.446277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 226/454] 10.222.0.26 () {56 vars in 1180 bytes} [Mon Jul 11 11:05:41 2022] GET /identity/v3/groups/558475b058dd446f84d69c072ec0e144 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:41.452602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8e394-ca73-44a6-9e56-9e2d3382830d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:41.551395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8e394-ca73-44a6-9e56-9e2d3382830d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:41.582764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fce895-5b9f-4107-b4fb-0b4e01f88d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:41.583433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fce895-5b9f-4107-b4fb-0b4e01f88d55 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:41.583718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fce895-5b9f-4107-b4fb-0b4e01f88d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:41.583996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fce895-5b9f-4107-b4fb-0b4e01f88d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:41.584824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fce895-5b9f-4107-b4fb-0b4e01f88d55 None admin] RBAC: Adding query filter params (group.id=558475b058dd446f84d69c072ec0e144, role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=d16c61acfee74652b6cd4c256cd8f443) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:41.585135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fce895-5b9f-4107-b4fb-0b4e01f88d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:41.586542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fce895-5b9f-4107-b4fb-0b4e01f88d55 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:41.592740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 229/455] 10.222.0.26 () {56 vars in 1376 bytes} [Mon Jul 11 11:05:41 2022] GET /identity/v3/role_assignments?group.id=558475b058dd446f84d69c072ec0e144&scope.project.id=d16c61acfee74652b6cd4c256cd8f443&role.id=007af6bd1b734517b934b4cac45faf2c => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:44.033407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8908ed-5520-4c93-9d74-9b30df2faa66 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.034441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8908ed-5520-4c93-9d74-9b30df2faa66 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.034935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8908ed-5520-4c93-9d74-9b30df2faa66 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.036860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 227/456] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:44.042374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464154-b910-41e4-871c-56d689a9cfc2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.043361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464154-b910-41e4-871c-56d689a9cfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.043853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464154-b910-41e4-871c-56d689a9cfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.068518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e1464154-b910-41e4-871c-56d689a9cfc2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:44.070047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e1464154-b910-41e4-871c-56d689a9cfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:44.104781 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:44.163966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 230/457] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:44.204935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8908ed-5520-4c93-9d74-9b30df2faa66 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:44.236520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26723-d8e1-418f-92ba-e6ed34b0accb None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.237155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26723-d8e1-418f-92ba-e6ed34b0accb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.237656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26723-d8e1-418f-92ba-e6ed34b0accb None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.240195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 231/458] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:44.246896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9290f-5b75-4144-81d0-a4a43c403a7d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.247770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9290f-5b75-4144-81d0-a4a43c403a7d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.248253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9290f-5b75-4144-81d0-a4a43c403a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.273348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b4d9290f-5b75-4144-81d0-a4a43c403a7d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:44.274918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b4d9290f-5b75-4144-81d0-a4a43c403a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:44.374403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 232/459] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:44.398297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8908ed-5520-4c93-9d74-9b30df2faa66 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:44.429563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a05a65-955c-462f-9ebc-628f2170fe13 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.430347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a05a65-955c-462f-9ebc-628f2170fe13 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.432073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a05a65-955c-462f-9ebc-628f2170fe13 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.435115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 233/460] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:44.436294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a05a65-955c-462f-9ebc-628f2170fe13 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:44.449543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898ef27-b9e8-4b11-b09e-a14f7bdc8224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:44.450339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ef27-b9e8-4b11-b09e-a14f7bdc8224 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.450918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ef27-b9e8-4b11-b09e-a14f7bdc8224 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.451197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898ef27-b9e8-4b11-b09e-a14f7bdc8224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.463454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898ef27-b9e8-4b11-b09e-a14f7bdc8224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:44.465420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898ef27-b9e8-4b11-b09e-a14f7bdc8224 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f898ef27-b9e8-4b11-b09e-a14f7bdc8224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:44.487913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:44.488835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:44.489702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:44.489702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:44.489702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:44.489702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:44.489702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 228/461] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:44.490823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e596e-5dab-4744-bb2f-5821b9c698ad None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.491282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e596e-5dab-4744-bb2f-5821b9c698ad None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.491763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e596e-5dab-4744-bb2f-5821b9c698ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.512033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-488e596e-5dab-4744-bb2f-5821b9c698ad None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:44.513391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-488e596e-5dab-4744-bb2f-5821b9c698ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:44.543745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:44.549472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:44.555734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a05a65-955c-462f-9ebc-628f2170fe13 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:44.591600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:44.592213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 229/462] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:44.593826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.593943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e596e-5dab-4744-bb2f-5821b9c698ad None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:44.594374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.594878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.619072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:44.622382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:44.652371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:44.653406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:44.654290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:44.654290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:44.654290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:44.654290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:44.656472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 234/463] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:44.657510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:44.702146 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e596e-5dab-4744-bb2f-5821b9c698ad None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:44.737943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:44.738740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.739098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.739467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.740277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:44.740874 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:44.742086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:44.754772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 230/464] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:44.755828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:44.764580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bd389-f320-48ac-add8-24a4b28bc512 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:44.781366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:44.801056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:44.802205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.802660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.803157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.814963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:44.816721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:44.818705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:44.836519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:44.837455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:44.838329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:44.838329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:44.838329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:44.838329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:44.845701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 235/465] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:44.846811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:44.860563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411bc5f-483e-4fca-9b12-8d935fead471 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:44.901162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:44.902361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.902714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.902931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.903723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:44.904019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:44.905409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:44.918994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 231/466] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:44.919866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:44.958280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bc185-6fc8-4a1c-a3d9-64979fb8e715 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:44.996684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef59a7-1062-43ae-9820-8bd7393210dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:44.997356 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef59a7-1062-43ae-9820-8bd7393210dd None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:44.997643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef59a7-1062-43ae-9820-8bd7393210dd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:44.997921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef59a7-1062-43ae-9820-8bd7393210dd None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:44.998626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef59a7-1062-43ae-9820-8bd7393210dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:45.000151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef59a7-1062-43ae-9820-8bd7393210dd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:45.010979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:45.029392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc99ba-5415-4376-8f80-57f4f0c6cef8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:45.062844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:45.063528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.063817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.064096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.064820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:45.065131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:45.066507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:45.078450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 232/467] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:45.079290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:45.111754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 236/468] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:44 2022] POST /identity/v3/users => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:45.113099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb59ec-2404-4790-a712-29c06c64cab0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.113552 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb59ec-2404-4790-a712-29c06c64cab0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.113736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb59ec-2404-4790-a712-29c06c64cab0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.115098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 237/469] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:45.115812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb59ec-2404-4790-a712-29c06c64cab0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:45.196769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:45.210819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169fbdc-a447-4113-8d55-81b13de86e4c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:45.258017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc617837-5465-45c6-b04a-7cb892b33d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:45.259097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc617837-5465-45c6-b04a-7cb892b33d08 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.259629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc617837-5465-45c6-b04a-7cb892b33d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.260117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc617837-5465-45c6-b04a-7cb892b33d08 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.260835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc617837-5465-45c6-b04a-7cb892b33d08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:45.262949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc617837-5465-45c6-b04a-7cb892b33d08 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:45.266963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb59ec-2404-4790-a712-29c06c64cab0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:45.303556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07ea13-0e33-47b9-8afb-d7b068669e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:45.304218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07ea13-0e33-47b9-8afb-d7b068669e51 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.304498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07ea13-0e33-47b9-8afb-d7b068669e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.304770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07ea13-0e33-47b9-8afb-d7b068669e51 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.305468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07ea13-0e33-47b9-8afb-d7b068669e51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:45.306888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07ea13-0e33-47b9-8afb-d7b068669e51 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:45.332775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:45.338296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 233/470] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:45 2022] POST /identity/v3/users => generated 248 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:45.340225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aae55-c0eb-43a5-8cd7-717d798954f1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.340520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aae55-c0eb-43a5-8cd7-717d798954f1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.340802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aae55-c0eb-43a5-8cd7-717d798954f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.359076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3b7aae55-c0eb-43a5-8cd7-717d798954f1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:45.360358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3b7aae55-c0eb-43a5-8cd7-717d798954f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:45.429573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 238/471] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:45 2022] POST /identity/v3/users => generated 248 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:45.437527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079195d-450c-48cd-8cb5-77ef2cb6e2d8 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.438110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079195d-450c-48cd-8cb5-77ef2cb6e2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.438574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079195d-450c-48cd-8cb5-77ef2cb6e2d8 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.450875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 239/472] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:45.453390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ccf3-247d-4666-ac33-5f9b259efdfc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.454126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ccf3-247d-4666-ac33-5f9b259efdfc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.454612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6ccf3-247d-4666-ac33-5f9b259efdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.455629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 234/473] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:45.474122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-59a6ccf3-247d-4666-ac33-5f9b259efdfc None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:45.475772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-59a6ccf3-247d-4666-ac33-5f9b259efdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:45.491841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aae55-c0eb-43a5-8cd7-717d798954f1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:45.589073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 240/474] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:45.624450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6ccf3-247d-4666-ac33-5f9b259efdfc None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:45.624675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aae55-c0eb-43a5-8cd7-717d798954f1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:45.656576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:45.664611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:45.665420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.665703 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.665978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.673826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:45.678648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:45.680567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:45.702367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:45.703305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:45.704169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:45.704169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:45.704169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:45.704169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:45.704169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 235/475] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:45.709432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:45.752161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6ccf3-247d-4666-ac33-5f9b259efdfc None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:45.793694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:45.794471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.794788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.795129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.796032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:45.797525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:45.813873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1aa51-a630-46c7-b9ab-11185294534a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:45.834840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.sql.core [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 11:05:45.834840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 11 11:05:45.834840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [parameters: {'id': '1b8cc995e5d749e598b18e3670bc6ef7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 11 11:05:45.834840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80499) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 11:05:45.851615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:45.852342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:45.852628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:45.852954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:45.853700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:45.854008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result.read() Jul 11 11:05:45.855240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return ref.to_dict() Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:05:45.856364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application yield resource Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise self.value Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application session.commit() Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 11:05:45.857492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.session.flush() Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application compat.raise_( Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 11:05:45.858486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:05:45.859378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result.read() Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:05:45.860372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [parameters: {'id': '1b8cc995e5d749e598b18e3670bc6ef7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:45.861205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 11 11:05:45.862289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:45.863373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:45.867383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 241/476] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 11 11:05:45 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:45.868505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 236/477] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:45.871647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:45.873952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:45.919082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:45.986537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:45.989843 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f52f72-649c-4edb-a9d2-e15cafc48ccc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:46.003387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85503745-c9e2-4ebb-bc9c-79c87adc6e9b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:46.026450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62534d55-b80f-4a08-b36b-430dae57369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:46.027603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62534d55-b80f-4a08-b36b-430dae57369e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:46.028239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62534d55-b80f-4a08-b36b-430dae57369e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:46.028663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62534d55-b80f-4a08-b36b-430dae57369e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:46.040505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62534d55-b80f-4a08-b36b-430dae57369e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-62534d55-b80f-4a08-b36b-430dae57369e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:46.055195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:46.056171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 11:05:46.057053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:46.063351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 242/478] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:05:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:46.065565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62534d55-b80f-4a08-b36b-430dae57369e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:46.079957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d6fb6-5e6f-4607-a954-12b194aefff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:46.080785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d6fb6-5e6f-4607-a954-12b194aefff2 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:46.081311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d6fb6-5e6f-4607-a954-12b194aefff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:46.081543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d6fb6-5e6f-4607-a954-12b194aefff2 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:46.082372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d6fb6-5e6f-4607-a954-12b194aefff2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:46.083949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d6fb6-5e6f-4607-a954-12b194aefff2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:46.182688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62534d55-b80f-4a08-b36b-430dae57369e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:46.206726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 237/479] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:45 2022] POST /identity/v3/users => generated 249 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:05:46.221543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb85f6b-28d4-4d90-8aaa-92bd3514a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:46.222503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85f6b-28d4-4d90-8aaa-92bd3514a667 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:46.222989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85f6b-28d4-4d90-8aaa-92bd3514a667 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:46.223453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb85f6b-28d4-4d90-8aaa-92bd3514a667 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:46.224388 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb85f6b-28d4-4d90-8aaa-92bd3514a667 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:46.224900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb85f6b-28d4-4d90-8aaa-92bd3514a667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:46.226461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb85f6b-28d4-4d90-8aaa-92bd3514a667 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:46.236387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 243/480] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:47.633118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0907c-6802-4e1e-8b21-0fbfd8b43cb7 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:47.634142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0907c-6802-4e1e-8b21-0fbfd8b43cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:47.634654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0907c-6802-4e1e-8b21-0fbfd8b43cb7 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:47.636583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 238/481] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:47.641468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813ea34-d646-44d5-9154-19e8da5a82c2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:47.642275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813ea34-d646-44d5-9154-19e8da5a82c2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:47.642755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9813ea34-d646-44d5-9154-19e8da5a82c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:47.662145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9813ea34-d646-44d5-9154-19e8da5a82c2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:47.663810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9813ea34-d646-44d5-9154-19e8da5a82c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:47.684717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:47.738378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 244/482] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:47.785563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0907c-6802-4e1e-8b21-0fbfd8b43cb7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:47.913579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:47.924537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0907c-6802-4e1e-8b21-0fbfd8b43cb7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:47.965353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea05339-e3cf-4ac8-a7c9-e3a5ade5a49f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:47.965661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea05339-e3cf-4ac8-a7c9-e3a5ade5a49f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:47.965920 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea05339-e3cf-4ac8-a7c9-e3a5ade5a49f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:47.967311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 245/483] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:47.972528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edcb5-04bd-4d39-9d4e-71ad473c8cfd None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:47.972818 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edcb5-04bd-4d39-9d4e-71ad473c8cfd None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:47.973100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40edcb5-04bd-4d39-9d4e-71ad473c8cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:47.984825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc47243-5a4d-4644-ac76-0f1ced835cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:47.985942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc47243-5a4d-4644-ac76-0f1ced835cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:47.986432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc47243-5a4d-4644-ac76-0f1ced835cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:47.986982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc47243-5a4d-4644-ac76-0f1ced835cd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:47.993002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f40edcb5-04bd-4d39-9d4e-71ad473c8cfd None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:47.994239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f40edcb5-04bd-4d39-9d4e-71ad473c8cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:48.005549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc47243-5a4d-4644-ac76-0f1ced835cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-bbc47243-5a4d-4644-ac76-0f1ced835cd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:48.014176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:48.015079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:48.015942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:48.018719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 239/484] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:48.024966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc47243-5a4d-4644-ac76-0f1ced835cd9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.104048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 246/485] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:48.148689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40edcb5-04bd-4d39-9d4e-71ad473c8cfd None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.192545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc47243-5a4d-4644-ac76-0f1ced835cd9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.244162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc246daa-b521-4699-9275-9b724551e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.244849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc246daa-b521-4699-9275-9b724551e5bc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.245135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc246daa-b521-4699-9275-9b724551e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc246daa-b521-4699-9275-9b724551e5bc None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.246232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc246daa-b521-4699-9275-9b724551e5bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:48.246546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc246daa-b521-4699-9275-9b724551e5bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:48.248757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc246daa-b521-4699-9275-9b724551e5bc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:48.258441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 240/486] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:48.260109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8745-08d0-48f8-879f-45d7c6a86c99 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.260607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8745-08d0-48f8-879f-45d7c6a86c99 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.261612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca8745-08d0-48f8-879f-45d7c6a86c99 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.267050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 241/487] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:48.268083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca8745-08d0-48f8-879f-45d7c6a86c99 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.289348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:48.290863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:48.312138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40edcb5-04bd-4d39-9d4e-71ad473c8cfd None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.347472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d6c3f-dc73-47ea-9d38-3d27f0a677f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.348273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d6c3f-dc73-47ea-9d38-3d27f0a677f5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.348561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d6c3f-dc73-47ea-9d38-3d27f0a677f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.349497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d6c3f-dc73-47ea-9d38-3d27f0a677f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.359809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d6c3f-dc73-47ea-9d38-3d27f0a677f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c79d6c3f-dc73-47ea-9d38-3d27f0a677f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:48.367807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:48.368908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:48.369907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:48.373304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 247/488] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:48.374633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da44bc-aace-4963-a739-43010fb7ca18 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.375021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da44bc-aace-4963-a739-43010fb7ca18 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.375446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da44bc-aace-4963-a739-43010fb7ca18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.387959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca8745-08d0-48f8-879f-45d7c6a86c99 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.393584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-84da44bc-aace-4963-a739-43010fb7ca18 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:48.394967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-84da44bc-aace-4963-a739-43010fb7ca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:48.425114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.426142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.426596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.427018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.438632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:48.440333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:48.459198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:48.460304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:48.461286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:48.461286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:48.461286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:48.461286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:48.463519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 242/489] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:48.465004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.475522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 248/490] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:48.476465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da44bc-aace-4963-a739-43010fb7ca18 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.539862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:48.552519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:48.574813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629531e-4da6-4c90-81aa-04f31a8c983e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.583928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da44bc-aace-4963-a739-43010fb7ca18 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.616836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.617959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.618428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.618967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.620047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:48.620670 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:48.622230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:48.630961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.631739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.632144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.632550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.632739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 243/491] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:48.633280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:48.633659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:48.634136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.635188 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:48.654464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 249/492] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:48.659453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.748567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a112f20-19e2-4878-a7ef-372bbdda6eb2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.764569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ff18a-8b36-42fa-a5da-fc4cfefa71ab None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.777699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:48.782548 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47318ad2-9f74-4164-a456-f102e43a5788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.783373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47318ad2-9f74-4164-a456-f102e43a5788 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.783661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47318ad2-9f74-4164-a456-f102e43a5788 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.783941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47318ad2-9f74-4164-a456-f102e43a5788 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.795001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47318ad2-9f74-4164-a456-f102e43a5788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:48.801281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-47318ad2-9f74-4164-a456-f102e43a5788 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:48.802947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:48.804007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:48.805088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 244/493] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:48.805946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47318ad2-9f74-4164-a456-f102e43a5788 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.811676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe77da-fef2-4fc4-b22f-3610483454ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.812506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe77da-fef2-4fc4-b22f-3610483454ca None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.812796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe77da-fef2-4fc4-b22f-3610483454ca None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.813075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe77da-fef2-4fc4-b22f-3610483454ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.823995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe77da-fef2-4fc4-b22f-3610483454ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:48.825492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe77da-fef2-4fc4-b22f-3610483454ca None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4afe77da-fef2-4fc4-b22f-3610483454ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:48.842878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:48.844544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:48.845413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:48.845413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:48.845413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:48.845413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:48.845413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 250/494] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:48.845693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf673ac-01d8-4d76-95d6-dfeb86042669 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.845693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf673ac-01d8-4d76-95d6-dfeb86042669 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.846226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf673ac-01d8-4d76-95d6-dfeb86042669 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.847404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 251/495] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:48.848040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf673ac-01d8-4d76-95d6-dfeb86042669 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.912768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47318ad2-9f74-4164-a456-f102e43a5788 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.947572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf81e1f-e832-4145-9e0c-68aa10595e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.948360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf81e1f-e832-4145-9e0c-68aa10595e2a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.948645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf81e1f-e832-4145-9e0c-68aa10595e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.948920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf81e1f-e832-4145-9e0c-68aa10595e2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.954927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf673ac-01d8-4d76-95d6-dfeb86042669 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:48.956010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf81e1f-e832-4145-9e0c-68aa10595e2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:48.957461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf81e1f-e832-4145-9e0c-68aa10595e2a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-aaf81e1f-e832-4145-9e0c-68aa10595e2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:48.973287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:48.974217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:48.975145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 245/496] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:48.976244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7288874-1cb8-4581-ad98-1bb20de68c15 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.976244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7288874-1cb8-4581-ad98-1bb20de68c15 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.976393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7288874-1cb8-4581-ad98-1bb20de68c15 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.977833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 246/497] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:48.978422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7288874-1cb8-4581-ad98-1bb20de68c15 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:48.994048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdb115-39bb-4e41-9ab4-d8947a07f3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:48.994705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdb115-39bb-4e41-9ab4-d8947a07f3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:48.994992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdb115-39bb-4e41-9ab4-d8947a07f3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:48.995289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdb115-39bb-4e41-9ab4-d8947a07f3a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:48.996608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdb115-39bb-4e41-9ab4-d8947a07f3a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:48.996913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdb115-39bb-4e41-9ab4-d8947a07f3a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:48.998267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdb115-39bb-4e41-9ab4-d8947a07f3a3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.006184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 252/498] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:49.007277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064bb7-9047-4236-8f5a-2d2ea4db1e31 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.007575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064bb7-9047-4236-8f5a-2d2ea4db1e31 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.007850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064bb7-9047-4236-8f5a-2d2ea4db1e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.023127 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e3064bb7-9047-4236-8f5a-2d2ea4db1e31 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:49.024324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e3064bb7-9047-4236-8f5a-2d2ea4db1e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:49.036935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.067844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.084780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7288874-1cb8-4581-ad98-1bb20de68c15 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.091285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 253/499] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:49.092120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064bb7-9047-4236-8f5a-2d2ea4db1e31 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.118932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59256a12-5b4d-4881-b4ff-e868f2250749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.119624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59256a12-5b4d-4881-b4ff-e868f2250749 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.119909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59256a12-5b4d-4881-b4ff-e868f2250749 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.120189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59256a12-5b4d-4881-b4ff-e868f2250749 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.121485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59256a12-5b4d-4881-b4ff-e868f2250749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:49.121798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59256a12-5b4d-4881-b4ff-e868f2250749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:49.123172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59256a12-5b4d-4881-b4ff-e868f2250749 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.134257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 247/500] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:49.135299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4fb9d-153d-4b21-af80-6d9375cbfeae None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.135594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4fb9d-153d-4b21-af80-6d9375cbfeae None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.135867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4fb9d-153d-4b21-af80-6d9375cbfeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.151271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bfe4fb9d-153d-4b21-af80-6d9375cbfeae None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:49.152495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bfe4fb9d-153d-4b21-af80-6d9375cbfeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:49.191725 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064bb7-9047-4236-8f5a-2d2ea4db1e31 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.216679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 248/501] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:49.217501 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4fb9d-153d-4b21-af80-6d9375cbfeae None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.230739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.231588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.231874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.232149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.233005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:49.233319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:49.234772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.266411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.275349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 254/502] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:49.276187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.310578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.326382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4fb9d-153d-4b21-af80-6d9375cbfeae None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.359362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.360154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.360441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.360712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.370514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:49.371974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.374372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8acf6d-2e9c-4720-81ca-cbee846fd518 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:49.389592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:49.390452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:49.391371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:49.391371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:49.391371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:49.391371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:49.391371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 249/503] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:49.391639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.410243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b27f5d-fb9a-47ca-8bf4-c4fc58929c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.411044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b27f5d-fb9a-47ca-8bf4-c4fc58929c28 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.411356 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b27f5d-fb9a-47ca-8bf4-c4fc58929c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.411634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b27f5d-fb9a-47ca-8bf4-c4fc58929c28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.421811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b27f5d-fb9a-47ca-8bf4-c4fc58929c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-44b27f5d-fb9a-47ca-8bf4-c4fc58929c28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:49.429701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:49.430611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 255/504] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:49.431759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b27f5d-fb9a-47ca-8bf4-c4fc58929c28 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.487633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.493598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ae002-fe82-433a-bd01-0cc25db6bd8f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.522530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.526906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.527743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.528031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.528309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] PATH_INFO: `/v3/users/glance` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.528894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b27f5d-fb9a-47ca-8bf4-c4fc58929c28 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.536284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:49.537873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:49.552880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:49.554690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 250/505] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:49.556450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.572541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13972190-71ee-4be0-a544-358b39920583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.573330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972190-71ee-4be0-a544-358b39920583 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.574144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972190-71ee-4be0-a544-358b39920583 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.574424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972190-71ee-4be0-a544-358b39920583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.585929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972190-71ee-4be0-a544-358b39920583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:49.587431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972190-71ee-4be0-a544-358b39920583 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-13972190-71ee-4be0-a544-358b39920583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:49.604721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:49.605681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:49.606532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:49.606532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:49.606532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:49.606532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:49.606532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 256/506] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:49.606792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13972190-71ee-4be0-a544-358b39920583 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.662001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c8337-56c4-4b65-be65-45795fad55ca None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.694567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0f3db-323d-4ff6-b026-e87b24518b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.695200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0f3db-323d-4ff6-b026-e87b24518b14 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.695510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0f3db-323d-4ff6-b026-e87b24518b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.695785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0f3db-323d-4ff6-b026-e87b24518b14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.702254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13972190-71ee-4be0-a544-358b39920583 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.719378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.736100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.736756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.737035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.737308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.738033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:49.738338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:49.739782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.752247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 257/507] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:49.753055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.761678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.767554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0f3db-323d-4ff6-b026-e87b24518b14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:49.769137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0f3db-323d-4ff6-b026-e87b24518b14 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.851699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dce569-afd2-429f-bc83-717b3456e4c8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.865080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 251/508] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:49.865912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0f3db-323d-4ff6-b026-e87b24518b14 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.897590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:49.898257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:49.898787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:49.899060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:49.899845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:49.900152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:49.901619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:49.910425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 258/509] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:49.911232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:49.926559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:49.971259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0f3db-323d-4ff6-b026-e87b24518b14 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:49.974832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.004337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.005004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.005292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.005571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.006407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:50.006719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.007263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b8e60-ee1e-40af-9322-01d8ace5a2b6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.008172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.041189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.041846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.042143 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.042427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.043141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:50.043479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.044016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 252/510] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:50.044832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.044932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.060573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 259/511] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:50.061399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.147060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c38f22-e7f5-404a-a138-505ecfc7e277 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.158267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae491fcd-4ff1-4c5e-99e3-4b4984358c4b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.159961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.180442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.181223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.181503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.181796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.184555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.188999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.190457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.192015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.192804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.193134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.193411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.203039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.204670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:50.205471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:50.206553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 253/512] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:50.207520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:50.232933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:50.234029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:50.236478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:50.236478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:50.236478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:50.236478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:50.236478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 260/513] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:50.236478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.307141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241eeb24-eb34-4fb1-80c4-780b73ad4c40 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.330438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5ae1a-e7a6-47ce-a8ca-eee8f47cc84c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.340354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.341141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.341429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.341708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.351558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.352994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.363474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8de0-a9e5-46b5-87a4-8820797a9b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.364100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8de0-a9e5-46b5-87a4-8820797a9b5b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.364381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8de0-a9e5-46b5-87a4-8820797a9b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.364654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8de0-a9e5-46b5-87a4-8820797a9b5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:50.371279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:50.372137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:50.373164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:50.373164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:50.373164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:50.373164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:50.373164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 254/514] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:50.373839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.396775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.413363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.432606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f8de0-a9e5-46b5-87a4-8820797a9b5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.434625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f8de0-a9e5-46b5-87a4-8820797a9b5b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.471177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ff3f9-f39b-4312-9635-336c7aa64f5e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.504695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524696b-59e2-4906-810a-ca2bf3a567ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.505360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524696b-59e2-4906-810a-ca2bf3a567ae None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.505645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524696b-59e2-4906-810a-ca2bf3a567ae None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.505921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524696b-59e2-4906-810a-ca2bf3a567ae None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.506623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524696b-59e2-4906-810a-ca2bf3a567ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.508074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524696b-59e2-4906-810a-ca2bf3a567ae None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.529730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 261/515] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:50.530558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8de0-a9e5-46b5-87a4-8820797a9b5b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.610066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.627824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8de0-a9e5-46b5-87a4-8820797a9b5b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.628340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 255/516] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:05:50 2022] POST /identity/v3/users => generated 247 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:05:50.628968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524696b-59e2-4906-810a-ca2bf3a567ae None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.667798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.668517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.668803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.669081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.669943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:50.670261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.671774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.692961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 262/517] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:50.693839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.727474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.751593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524696b-59e2-4906-810a-ca2bf3a567ae None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.790985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.791785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.792078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.792922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.793658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:50.793971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.795380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.809691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 256/518] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:50.809849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.810318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492a312-e9ec-4301-a7d7-37be9229643c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.857457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.858658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.858954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.859395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.860259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:50.860571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.862672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.870732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.879304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 263/519] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:50.879304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:50.918396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b9a16-a10e-4365-a185-cb942e13219a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.954995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:50.955837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:50.956122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:50.956396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:50.966332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:50.968739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:50.970209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:50.984030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd84072-7e9f-4546-b229-28dc539ab98d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:50.987684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:50.988537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:50.989390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:50.989390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:50.989390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:50.989390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:50.993683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 257/520] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:50.994407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.021354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f569c-9070-465d-87bc-50339cdffa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.022010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f569c-9070-465d-87bc-50339cdffa51 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.022297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f569c-9070-465d-87bc-50339cdffa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.022577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f569c-9070-465d-87bc-50339cdffa51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.092184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:51.095806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f569c-9070-465d-87bc-50339cdffa51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.097337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f569c-9070-465d-87bc-50339cdffa51 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.112936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2b52a-1dc7-4ca6-988f-774424cfa325 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.152263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.153057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.153342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.153624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.160181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.161617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:51.176593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:51.177749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 258/521] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:51.178601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.209230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 264/522] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:51.210406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f569c-9070-465d-87bc-50339cdffa51 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.235970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:51.289239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125c200-a717-48d9-954c-86f3c4670fd1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.317845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:51.321937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f569c-9070-465d-87bc-50339cdffa51 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.324194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.325066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.325352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.328332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.333304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.334741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:51.349560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:51.350484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:51.351505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 259/523] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:51.352102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.363786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.364446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.364730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.365005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.365737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:51.366044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.367452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.379001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 265/524] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:51.379001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.459349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f43300-0848-442e-871b-874d0aae67b8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.472288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:51.489259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a507220-b9fa-4068-9464-7f035e4b7263 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.494641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.495388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.495646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.495973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.496817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:51.497137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.498599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.520999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 260/525] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:51.521663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.523794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.524584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.524867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.525143 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.536907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.538434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:51.556817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:51.557822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:51.558677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:51.558677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:51.558677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:51.558677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:51.558677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 266/526] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:51.558954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.572061 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:51.633912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94633f71-1bfb-4402-bbc4-7aa2eaf3894d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.660451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414b155-389b-4f5d-a799-37a6a44b0a54 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.673791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.674465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.675006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.675310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.676151 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:51.676466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.677904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.695059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02091623-0ef8-489e-b389-026115057ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.695894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02091623-0ef8-489e-b389-026115057ffa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.696178 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02091623-0ef8-489e-b389-026115057ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.696451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02091623-0ef8-489e-b389-026115057ffa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.701453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 261/527] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:51.701453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.708202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02091623-0ef8-489e-b389-026115057ffa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.709719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02091623-0ef8-489e-b389-026115057ffa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-02091623-0ef8-489e-b389-026115057ffa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:51.725109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:51.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:51.726814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 267/528] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:51.727889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02091623-0ef8-489e-b389-026115057ffa None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.737380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:51.802552 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:51.810294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ec7b9-ac1b-41b5-96af-58ab33a40f83 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.841380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02091623-0ef8-489e-b389-026115057ffa None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:51.848524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.849182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.849805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.849805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.850568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=258ce5f4d6ff400199a80157cc34f5e5) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:51.850882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.852353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.858487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 262/529] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/role_assignments?user.id=258ce5f4d6ff400199a80157cc34f5e5&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:51.859283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.898068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:51.899120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:51.899933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:51.901478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:51.903076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:51.903610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:51.905249 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:51.923702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 268/530] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:51.925540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:51.983158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:51.996547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ec763-ce41-4a61-9b46-44e9dfc4d330 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:52.039665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282fc16-862f-4aed-bdb3-c7161fd44760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:52.040317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282fc16-862f-4aed-bdb3-c7161fd44760 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.040604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282fc16-862f-4aed-bdb3-c7161fd44760 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.040887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282fc16-862f-4aed-bdb3-c7161fd44760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.074739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ce86-b09b-42b0-97b1-a31b5f9e7aed None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:52.113016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:52.121923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:52.122962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.123715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.123715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.123927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282fc16-862f-4aed-bdb3-c7161fd44760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:52.124764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:52.125214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:52.125878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282fc16-862f-4aed-bdb3-c7161fd44760 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:52.126810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:52.152598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 269/531] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:52.153780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:52.240868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:52.241154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 263/532] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:52.242243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282fc16-862f-4aed-bdb3-c7161fd44760 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:52.279937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e013ea-c66b-467a-b076-174d5e82bfef None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:52.316906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:52.317712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.317997 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.318274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.325442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:52.326917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:52.341544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:52.342461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:52.343351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:52.343351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:52.343351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:52.343351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:52.343351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:52.343351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:52.343351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:52.343351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:52.349294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 270/533] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:52.350185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:52.360654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282fc16-862f-4aed-bdb3-c7161fd44760 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:52.397767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf865295-c163-4461-ac99-8718930af70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:52.398549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865295-c163-4461-ac99-8718930af70f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.398944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865295-c163-4461-ac99-8718930af70f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.399329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865295-c163-4461-ac99-8718930af70f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.400381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf865295-c163-4461-ac99-8718930af70f None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=e939bc73c40140cb89517e918722d154) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:52.400772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf865295-c163-4461-ac99-8718930af70f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:52.401260 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:52.403785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf865295-c163-4461-ac99-8718930af70f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:52.410209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 264/534] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:52 2022] GET /identity/v3/role_assignments?user.id=e939bc73c40140cb89517e918722d154&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:52.411122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf865295-c163-4461-ac99-8718930af70f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:52.456841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504e13-3674-4305-82f3-22903e89f437 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:52.492461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:52.493184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.493473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.493755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.506273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:52.506994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:52.507823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:52.526228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:52.530024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:52.531160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:52.531160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:52.531160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:52.531160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:52.531160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 271/535] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:52.533529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf865295-c163-4461-ac99-8718930af70f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:52.544361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:52.584288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:52.585363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.585840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.588177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.589450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:52.589985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:52.592930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:52.622997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 265/536] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:52.628930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:52.722074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1d57f-6c06-45a6-9f25-165fc5083135 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:52.733316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:52.792988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e40c70-c7db-436b-aa09-06a041384d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:52.793573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e40c70-c7db-436b-aa09-06a041384d23 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.793858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e40c70-c7db-436b-aa09-06a041384d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.794132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e40c70-c7db-436b-aa09-06a041384d23 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.794866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e40c70-c7db-436b-aa09-06a041384d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:52.795177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e40c70-c7db-436b-aa09-06a041384d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:52.796392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326090b8-edfa-4e47-aaa8-9afe4cce3690 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:52.796846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e40c70-c7db-436b-aa09-06a041384d23 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:52.809532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 272/537] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:52.815731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f175-42d3-4b2c-b802-49694f19bb3b None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.816029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f175-42d3-4b2c-b802-49694f19bb3b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.816312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35f175-42d3-4b2c-b802-49694f19bb3b None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.817884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 273/538] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:52.818234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:52.818719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35f175-42d3-4b2c-b802-49694f19bb3b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:52.838042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47863e8f-11fb-4c22-bafc-2178ec452e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:52.838699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47863e8f-11fb-4c22-bafc-2178ec452e68 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.839037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47863e8f-11fb-4c22-bafc-2178ec452e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.839430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47863e8f-11fb-4c22-bafc-2178ec452e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.840319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47863e8f-11fb-4c22-bafc-2178ec452e68 None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=d7cebda9ae4e4a6b9a9bbface3e7b232) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:52.840637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47863e8f-11fb-4c22-bafc-2178ec452e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:52.842248 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47863e8f-11fb-4c22-bafc-2178ec452e68 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:52.848786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 266/539] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:52 2022] GET /identity/v3/role_assignments?user.id=d7cebda9ae4e4a6b9a9bbface3e7b232&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:52.850092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f018-d29b-455a-9d8c-0da151a72664 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:52.850388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f018-d29b-455a-9d8c-0da151a72664 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:52.850669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f018-d29b-455a-9d8c-0da151a72664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:52.875297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0e45f018-d29b-455a-9d8c-0da151a72664 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:52.876646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0e45f018-d29b-455a-9d8c-0da151a72664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:52.993879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35f175-42d3-4b2c-b802-49694f19bb3b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:53.026053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 267/540] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:53.041019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:53.042753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:53.043345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:53.043816 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:53.052618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45f018-d29b-455a-9d8c-0da151a72664 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:53.057773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:53.059533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:53.076588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:53.077492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:53.078354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:53.078354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:53.078354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:53.078354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:53.078354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:53.078354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:53.078354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:53.078354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:53.081505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 274/541] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:53.088220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:53.138665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:53.204856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45f018-d29b-455a-9d8c-0da151a72664 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:53.224903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:53.267139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1ecb2-e54d-4961-93a5-454819c1967c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:53.268343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1ecb2-e54d-4961-93a5-454819c1967c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:53.268834 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1ecb2-e54d-4961-93a5-454819c1967c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:53.269311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1ecb2-e54d-4961-93a5-454819c1967c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:53.291244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1ecb2-e54d-4961-93a5-454819c1967c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:53.295345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ced39-8bba-40f3-bce1-0ec012965c66 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-72f1ecb2-e54d-4961-93a5-454819c1967c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:53.299087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:53.300242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:53.301049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:53.305349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 268/542] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:53.306562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1ecb2-e54d-4961-93a5-454819c1967c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:53.371948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:53.372618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:53.372910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:53.373191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:53.374154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:53.374474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:53.378050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:53.396082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 275/543] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:53.403981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:53.455601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1ecb2-e54d-4961-93a5-454819c1967c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:53.533291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:53.534239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:53.534669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:53.535101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:53.536016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:53.536487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:53.538013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:53.558783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 269/544] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:53.564044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:53.568558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:53.606101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036e010-efaf-4634-ba3d-1d4c1c9faf4f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:53.640975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:53.641868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:53.642298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:53.642846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:53.643936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=cb1fa1e11b1a44aaa31cc19f7c199285) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:53.644411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:53.646176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:53.652855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 276/545] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:53 2022] GET /identity/v3/role_assignments?user.id=cb1fa1e11b1a44aaa31cc19f7c199285&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:05:53.687654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:53.733054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91214904-07bd-442d-8dbe-2bdf213f3c08 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:53.771186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:53.772366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:53.772791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:53.773236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:53.785179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:53.786932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:53.806028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:53.808138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:53.809275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:53.809275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:53.809275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:53.809275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:53.811220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 270/546] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:53.820712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:53.963713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16484d98-5b15-4b08-83bf-ca159aac42f7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:54.006667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:54.008206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:54.008732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:54.009669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:54.010771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:54.011305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:54.012867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:54.025034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 277/547] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:54.031930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:54.176366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:54.196544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de22e6-5668-41ec-81aa-72ea5407d72d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:54.237518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadd843-0f48-42d5-91a1-154ec728ddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:54.238320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd843-0f48-42d5-91a1-154ec728ddc5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:54.238602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd843-0f48-42d5-91a1-154ec728ddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:54.238878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadd843-0f48-42d5-91a1-154ec728ddc5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:54.249634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadd843-0f48-42d5-91a1-154ec728ddc5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:54.251108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadd843-0f48-42d5-91a1-154ec728ddc5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6fadd843-0f48-42d5-91a1-154ec728ddc5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:54.271636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:54.272692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:05:54.273630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:54.278411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 271/548] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:05:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:54.283780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:54.302159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:54.376107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604acff-a08b-44c1-9c30-b55b3452e981 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:54.376426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604acff-a08b-44c1-9c30-b55b3452e981 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:54.376590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604acff-a08b-44c1-9c30-b55b3452e981 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:54.378113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 272/549] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:54.383340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77c334-92ec-49d7-8ecf-83d7236a4456 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:54.383630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77c334-92ec-49d7-8ecf-83d7236a4456 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:54.383908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77c334-92ec-49d7-8ecf-83d7236a4456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:54.411966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1b77c334-92ec-49d7-8ecf-83d7236a4456 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:54.413154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1b77c334-92ec-49d7-8ecf-83d7236a4456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:54.468371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28816de8-087c-4c54-8674-f09974c28418 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:54.531560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:54.532237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:54.532520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:54.532784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:54.533642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:54.533949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:54.535397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:54.541513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 273/550] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:05:54.567337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77c334-92ec-49d7-8ecf-83d7236a4456 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:54.568348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 278/551] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 11 11:05:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:54.576650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:54.675632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:54.699588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:54.738303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77c334-92ec-49d7-8ecf-83d7236a4456 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:54.746248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e734eef-8c36-4d56-ae4f-c6771bf92396 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:54.802782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c78a2-6950-45d3-9f10-d80ffe83db11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:54.803671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c78a2-6950-45d3-9f10-d80ffe83db11 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:54.803957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c78a2-6950-45d3-9f10-d80ffe83db11 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:54.804645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c78a2-6950-45d3-9f10-d80ffe83db11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:54.806331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd53911-815b-44ed-aac9-27db0902c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:54.806967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53911-815b-44ed-aac9-27db0902c133 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:54.807245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53911-815b-44ed-aac9-27db0902c133 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:54.807568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53911-815b-44ed-aac9-27db0902c133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:54.821669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c78a2-6950-45d3-9f10-d80ffe83db11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-997c78a2-6950-45d3-9f10-d80ffe83db11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:54.830009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:54.830923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:54.832497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 274/552] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:05:54.833191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236b314-47a5-43d4-8647-f42abc7319ef None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:54.833191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236b314-47a5-43d4-8647-f42abc7319ef None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:54.833514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236b314-47a5-43d4-8647-f42abc7319ef None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:54.838808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 275/553] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:54.838808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236b314-47a5-43d4-8647-f42abc7319ef None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:54.901888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd53911-815b-44ed-aac9-27db0902c133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:54.903365 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd53911-815b-44ed-aac9-27db0902c133 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:54.972571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236b314-47a5-43d4-8647-f42abc7319ef None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:54.987485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.020577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.021948 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.022459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.022955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.024365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:55.025166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:55.026816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:55.031972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 279/554] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:05:55.031972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc4a9d-b290-45fa-8742-4d3b6fe6ee6d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.031972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc4a9d-b290-45fa-8742-4d3b6fe6ee6d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.031972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc4a9d-b290-45fa-8742-4d3b6fe6ee6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.034558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.043950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 276/555] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:55.044920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.055714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-64bc4a9d-b290-45fa-8742-4d3b6fe6ee6d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:55.057276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-64bc4a9d-b290-45fa-8742-4d3b6fe6ee6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:55.133532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 280/556] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:55.144167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc4a9d-b290-45fa-8742-4d3b6fe6ee6d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.166671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d785-03bd-44f4-8997-ef2cbb14c5d3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.204688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c108e66-f2d2-4a2a-94e1-2c5567866508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.205480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c108e66-f2d2-4a2a-94e1-2c5567866508 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.205760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c108e66-f2d2-4a2a-94e1-2c5567866508 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.206039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c108e66-f2d2-4a2a-94e1-2c5567866508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.218060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c108e66-f2d2-4a2a-94e1-2c5567866508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:55.219649 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c108e66-f2d2-4a2a-94e1-2c5567866508 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7c108e66-f2d2-4a2a-94e1-2c5567866508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:55.238779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:55.239772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:55.240836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:55.240836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:55.240836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:55.240836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:55.240836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 277/557] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:55.241210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724038f1-d99b-4734-8838-b3ddc2b2eb13 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.241762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724038f1-d99b-4734-8838-b3ddc2b2eb13 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.241762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724038f1-d99b-4734-8838-b3ddc2b2eb13 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.243208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 278/558] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:55.247106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724038f1-d99b-4734-8838-b3ddc2b2eb13 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.250822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.257969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc4a9d-b290-45fa-8742-4d3b6fe6ee6d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.293902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.294689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.294974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.295272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.302629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.305487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:55.308072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:55.325764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:55.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:55.328068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:55.328068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:55.328068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:55.328068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:55.331611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 281/559] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:55.332259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.354500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724038f1-d99b-4734-8838-b3ddc2b2eb13 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.389243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10a61f-cee4-4186-8624-5841ee599253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.390060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a61f-cee4-4186-8624-5841ee599253 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.390348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a61f-cee4-4186-8624-5841ee599253 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.390625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a61f-cee4-4186-8624-5841ee599253 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.400805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10a61f-cee4-4186-8624-5841ee599253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5f10a61f-cee4-4186-8624-5841ee599253 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:55.409028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:55.410176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 279/560] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:55.416115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d2d5-ee1b-46c8-ba66-fc275e8e22d6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.417139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d2d5-ee1b-46c8-ba66-fc275e8e22d6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.417139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d2d5-ee1b-46c8-ba66-fc275e8e22d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.431771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8da8d2d5-ee1b-46c8-ba66-fc275e8e22d6 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:55.433485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8da8d2d5-ee1b-46c8-ba66-fc275e8e22d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:55.442912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff17e-7313-4592-aa84-9bda7823b28c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.481395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.482055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.482342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.482619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.483387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:55.483699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:55.485081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:55.501648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 282/561] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:55.501648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.508419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.513111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 280/562] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:55.514030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8d2d5-ee1b-46c8-ba66-fc275e8e22d6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.569331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.622515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb99b2-57f4-4465-aecb-f92b33c0ad3a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.631802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8d2d5-ee1b-46c8-ba66-fc275e8e22d6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.656161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.656830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.657121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.657403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.658128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:55.658442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:55.659864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:55.667474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.668145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.668427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.668703 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.669449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:55.669760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:55.671123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:55.673251 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 283/563] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:55.676192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.684904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 281/564] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:55.685997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.743483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.778503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e674bdb-3615-438d-ae3e-9bc3460e8515 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.789328 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25868d-4c8d-42a7-a81d-183067fa754a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.790851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.814722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.815594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.815879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.816159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.824002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:55.824897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ccc1-a87f-4d70-bb6b-25fc73f2d91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:55.825615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:55.825731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ccc1-a87f-4d70-bb6b-25fc73f2d91e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:55.826022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ccc1-a87f-4d70-bb6b-25fc73f2d91e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:55.826241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ccc1-a87f-4d70-bb6b-25fc73f2d91e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:55.839879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887ccc1-a87f-4d70-bb6b-25fc73f2d91e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:55.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:55.840924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:55.841832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 284/565] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:55.842346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6887ccc1-a87f-4d70-bb6b-25fc73f2d91e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:55.852338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:55.853301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:55.854178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:55.859418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 282/566] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:55.860171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ccc1-a87f-4d70-bb6b-25fc73f2d91e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:55.955186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac59666-aa33-423b-b697-cd0f15ddfa87 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.965579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887ccc1-a87f-4d70-bb6b-25fc73f2d91e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:55.985182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:55.998849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.000803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.000803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.000803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.003579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.004360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.004642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.004916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.007975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.009460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.014906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.016363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:56.025267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:56.026895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 285/567] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:56.027914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:56.037369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:56.038425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:56.039668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:56.039668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:56.039668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:56.039668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:56.039668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 283/568] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:56.039668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.057847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.133177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94a734-c545-4f34-980d-bffc28d8ee42 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.142209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14ea99-3dd7-4790-81c5-113309d3e08b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.165381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7f7a6-1c10-4d34-b7c4-d637b6fc5233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.166054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7f7a6-1c10-4d34-b7c4-d637b6fc5233 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.166337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7f7a6-1c10-4d34-b7c4-d637b6fc5233 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.166610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7f7a6-1c10-4d34-b7c4-d637b6fc5233 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.168015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7f7a6-1c10-4d34-b7c4-d637b6fc5233 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:56.168306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7f7a6-1c10-4d34-b7c4-d637b6fc5233 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.169907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7f7a6-1c10-4d34-b7c4-d637b6fc5233 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.176307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.176977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.177261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.177536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.178277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:56.178595 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.179977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.181644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 286/569] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:56.182870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5410027-1e83-4e98-8aa9-8c843909321b None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.183152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5410027-1e83-4e98-8aa9-8c843909321b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.183452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5410027-1e83-4e98-8aa9-8c843909321b None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.184656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 287/570] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:56.185366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5410027-1e83-4e98-8aa9-8c843909321b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.193185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 284/571] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:56.193831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.236376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.266980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.288304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5410027-1e83-4e98-8aa9-8c843909321b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.289349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171b714-447a-4b27-9b64-076bbba0bb8e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.321198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61c000-ebb7-4a3a-aff9-e558ad532c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.321857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c000-ebb7-4a3a-aff9-e558ad532c17 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.322142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c000-ebb7-4a3a-aff9-e558ad532c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.322418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c000-ebb7-4a3a-aff9-e558ad532c17 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.322822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.323814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61c000-ebb7-4a3a-aff9-e558ad532c17 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:56.324031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.324134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61c000-ebb7-4a3a-aff9-e558ad532c17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.324316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.324591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.325307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:56.325543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61c000-ebb7-4a3a-aff9-e558ad532c17 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.325645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.326943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.338248 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 285/572] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:56.339604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.346872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 288/573] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:56.348442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b7ab2-5942-4027-bdd9-d9ec9760c79a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.348737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b7ab2-5942-4027-bdd9-d9ec9760c79a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.349026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b7ab2-5942-4027-bdd9-d9ec9760c79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.365068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2a1b7ab2-5942-4027-bdd9-d9ec9760c79a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:56.366236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2a1b7ab2-5942-4027-bdd9-d9ec9760c79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:56.430821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 289/574] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:56.431808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b7ab2-5942-4027-bdd9-d9ec9760c79a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.443474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc181f-1a21-4214-b250-d6138a4e21a5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.465252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.478008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.479017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.479017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.479888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.480713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=7556ce4bf09d4113aec2037dd438cecb) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:56.481026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.482437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.485678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.488628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 286/575] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/role_assignments?user.id=7556ce4bf09d4113aec2037dd438cecb&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:56.489238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.552363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b7ab2-5942-4027-bdd9-d9ec9760c79a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.598125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.598388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30bf3c-a2a1-4ed2-9430-15560a8bf1b1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.598937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.599218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.599538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.611955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.613451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:56.633914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:56.634871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:56.636286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:56.636286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:56.636286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:56.636286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:56.636286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 290/576] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:56.636286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.642426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.643234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.643550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.643828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.652519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.653979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:56.668686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:56.669651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:56.670612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 287/577] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:56.671791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.730747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.738913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.745594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895e1f0-f8a2-4b01-bdb9-019f20107650 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.780618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.780851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df95cd-c86b-4cd8-85d8-55fa91f5fea7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.781416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.781697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.781973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.793035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.794480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:56.813001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:56.813887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:56.814787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:56.814787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:56.814787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:56.814787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:56.814787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 291/578] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:56.815796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.824819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4d8bf-3667-4b72-97f7-d23df6acc6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.825629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4d8bf-3667-4b72-97f7-d23df6acc6af None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.826198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4d8bf-3667-4b72-97f7-d23df6acc6af None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.826198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4d8bf-3667-4b72-97f7-d23df6acc6af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.837004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed4d8bf-3667-4b72-97f7-d23df6acc6af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-0ed4d8bf-3667-4b72-97f7-d23df6acc6af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:56.846472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:56.847616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 288/579] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:56.848479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4d8bf-3667-4b72-97f7-d23df6acc6af None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.922691 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3d866-0974-4e3c-b6d4-910543d45f5e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.958290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4d8bf-3667-4b72-97f7-d23df6acc6af None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:56.960703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.961371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.961655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.961934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.962669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:56.962978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.964909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:56.972706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.978913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 292/580] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:56.979648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:56.980591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:56.994757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:56.996069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:56.996375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:56.996642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:56.997472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:56.997785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:56.999233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.021025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 289/581] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:57.021025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.100786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d456009-17ec-47cc-b1d6-476f3a104acb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.139903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227e6ad-f7ad-4980-b1dd-af2bc0af14a0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.142796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.143367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.143367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.143553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.144156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:57.144467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.145822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.162638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 293/582] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:57.162638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.183234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.184031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.184415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.184762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.185577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:57.185959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.187488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.196051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 290/583] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:57.196861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.220990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:57.244691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:57.265152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95daa49-9fc6-4056-98e9-f9671702126c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.300908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.301697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.302032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.302313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.306603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccefa5-8b69-43aa-9553-ccda1fa14a15 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.309298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.310739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:57.326443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:57.327439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:05:57.328397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:57.331551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 294/584] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:57.332661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.347856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.348648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.348965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.349213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.360466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.361910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:57.379459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:57.380355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:57.386472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:57.386472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:57.386472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:57.386472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:57.386472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 291/585] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:57.386472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.444300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6906938-691c-4948-ba15-e9d6bdf0c7e2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.462999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:57.479750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.480537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.480943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.481158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.484987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:57.489511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.491412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.495931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9b4a-10f6-482b-88f6-2a7ca3b642d5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:57.506301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:57.507329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:57.508229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 295/586] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:57.508714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.533900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.534690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.534971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.535245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.546472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.547992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:57.569412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:57.570363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:57.571301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:57.571301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:57.571301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:57.571301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:57.571301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 292/587] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:57.571592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.629209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cd9d5-e698-442d-971c-52e42fc441ba None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.663575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.664246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.664537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.664815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.665664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:57.665978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.667991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.682174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d907a-1f0f-4a1b-9b5b-33b5028a0830 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.686403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 296/588] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:57.687434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.717498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.718279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.718569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.718959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.719711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:57.720025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.721507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.725843 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:57.734717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 293/589] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:57.734717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.740442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:57.793976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317e0c5-5636-40e2-ae1b-b29f6d461635 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.852806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.853468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.853754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.854079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.854884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:57.855237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.857153 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.876549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 297/590] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:57.876549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.880558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081cdfd-7528-49c2-b88a-ee03439e209d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.916992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:57.917653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:57.917938 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:57.918214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:57.918942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:57.919269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:57.920630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:57.933199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 294/591] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:57.934274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:57.981895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:57.988857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbab51-a291-459d-9026-638a8352cb42 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:57.999301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:58.029266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.030418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.030944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.031475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.042939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.044023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773cd23-8dd5-4d46-9ffd-d5aece3bc8a8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.044428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:58.064987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:58.065934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:58.066853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:58.066853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:58.066853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:58.066853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:58.066853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 298/592] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:58.067657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.086659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.087810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.088245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.088710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.095878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.097466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:58.112882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:58.114137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:58.115474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:58.115474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:58.115474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:58.115474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:58.115474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:58.115474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:58.115474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:58.115474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:58.117600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 295/593] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:58.119209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.183827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970ea9a-8c21-46a4-93eb-4823b0ea81cb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.220756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdb99f-d0a1-4e13-8461-1619112f3a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.221642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdb99f-d0a1-4e13-8461-1619112f3a4f None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.221924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdb99f-d0a1-4e13-8461-1619112f3a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.222201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdb99f-d0a1-4e13-8461-1619112f3a4f None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/258ce5f4d6ff400199a80157cc34f5e5/roles/d82425658c8545e6b546d601ef7b9b58` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.227592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a759b-8856-420e-8120-e79da2ff0197 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.237856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdb99f-d0a1-4e13-8461-1619112f3a4f None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=258ce5f4d6ff400199a80157cc34f5e5, role_id=d82425658c8545e6b546d601ef7b9b58)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.239382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdb99f-d0a1-4e13-8461-1619112f3a4f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.251459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:58.261541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.262328 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.262608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.262880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.269627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.271131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:05:58.285821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:58.286969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 296/594] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:58.288139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.337930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:58.351512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 299/595] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:58 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/258ce5f4d6ff400199a80157cc34f5e5/roles/d82425658c8545e6b546d601ef7b9b58 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:58.351512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdb99f-d0a1-4e13-8461-1619112f3a4f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.416536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8bfbd-ec47-4688-839c-7cddb1d1145b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.465340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdb99f-d0a1-4e13-8461-1619112f3a4f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.466259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.467354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.467740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.469104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.469969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:58.470383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.471941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.488843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 297/596] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:58.489947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.513877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927af562-d561-42c8-99b5-83681feade74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.514482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927af562-d561-42c8-99b5-83681feade74 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.514773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927af562-d561-42c8-99b5-83681feade74 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.515051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927af562-d561-42c8-99b5-83681feade74 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.515888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927af562-d561-42c8-99b5-83681feade74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:58.516195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927af562-d561-42c8-99b5-83681feade74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.518097 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927af562-d561-42c8-99b5-83681feade74 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.533990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 300/597] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:58.534854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927af562-d561-42c8-99b5-83681feade74 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.550087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:58.600863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:58.608513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015429b0-1bc0-4451-aa0a-4402b7595f3e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.642945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927af562-d561-42c8-99b5-83681feade74 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.643349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a8349-61c0-4b4d-9863-960aa14ef92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.644024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a8349-61c0-4b4d-9863-960aa14ef92a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.644316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a8349-61c0-4b4d-9863-960aa14ef92a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.644587 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a8349-61c0-4b4d-9863-960aa14ef92a None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.645421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a8349-61c0-4b4d-9863-960aa14ef92a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:58.645732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a8349-61c0-4b4d-9863-960aa14ef92a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.647150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a8349-61c0-4b4d-9863-960aa14ef92a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.664605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 298/598] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 11 11:05:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:58.665969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8a6ca-28c4-47e4-b9ba-d6f8357c8dec None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.666256 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8a6ca-28c4-47e4-b9ba-d6f8357c8dec None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.666532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8a6ca-28c4-47e4-b9ba-d6f8357c8dec None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.668441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 299/599] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:05:58.669154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8a6ca-28c4-47e4-b9ba-d6f8357c8dec None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.683976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.685199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.685732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.686233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.693664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.695548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:58.711447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:58.712361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:58.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:58.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:58.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:58.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:58.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:58.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:58.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:58.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:58.714447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 301/600] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:58.719367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.784988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8a6ca-28c4-47e4-b9ba-d6f8357c8dec None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.786908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:58.820689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19e59c-3c29-4979-b1d0-c73997781576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.822061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19e59c-3c29-4979-b1d0-c73997781576 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.822170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ed5a1-7b92-4840-a740-5abee7eba7fa None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:58.822357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19e59c-3c29-4979-b1d0-c73997781576 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.822613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19e59c-3c29-4979-b1d0-c73997781576 None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/e939bc73c40140cb89517e918722d154/roles/d82425658c8545e6b546d601ef7b9b58` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.841085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19e59c-3c29-4979-b1d0-c73997781576 None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=e939bc73c40140cb89517e918722d154, role_id=d82425658c8545e6b546d601ef7b9b58)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.841625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:58.842521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19e59c-3c29-4979-b1d0-c73997781576 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.857959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de498ec5-b21d-498a-ba7c-646e4704936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:58.858786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498ec5-b21d-498a-ba7c-646e4704936e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.859071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498ec5-b21d-498a-ba7c-646e4704936e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.859382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de498ec5-b21d-498a-ba7c-646e4704936e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.869064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de498ec5-b21d-498a-ba7c-646e4704936e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:58.870727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de498ec5-b21d-498a-ba7c-646e4704936e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-de498ec5-b21d-498a-ba7c-646e4704936e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:58.889250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:58.890436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:58.891700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:58.891700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:58.891700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:58.891700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:58.895113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 302/601] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:58.896256 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f4e1-69af-4258-a6dd-bc8437c6bb18 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:58.896543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f4e1-69af-4258-a6dd-bc8437c6bb18 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:58.896823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f4e1-69af-4258-a6dd-bc8437c6bb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:58.910057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 300/602] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:58 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/e939bc73c40140cb89517e918722d154/roles/d82425658c8545e6b546d601ef7b9b58 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:58.910812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19e59c-3c29-4979-b1d0-c73997781576 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:58.913368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d585f4e1-69af-4258-a6dd-bc8437c6bb18 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:05:58.915831 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d585f4e1-69af-4258-a6dd-bc8437c6bb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:05:59.003364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 303/603] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:05:59.003364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585f4e1-69af-4258-a6dd-bc8437c6bb18 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.060950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19e59c-3c29-4979-b1d0-c73997781576 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:59.100635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:59.101630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:59.101977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:59.102341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:59.104236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:59.104620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:59.106117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:59.115875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:59.118490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 301/604] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:59.119493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.121037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585f4e1-69af-4258-a6dd-bc8437c6bb18 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:59.139582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:59.175667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:59.176603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:59.177052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:59.177481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:59.178361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:59.178880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:59.180430 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:59.193802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 304/605] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:59.195335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.254771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffd1ea-daac-4abd-846f-3a4632c9caf7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:59.300944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b60085-99f5-4100-9046-09db623b8a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:59.301822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f0fb7-0b58-41a0-a721-9ec6a3014969 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:59.301946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60085-99f5-4100-9046-09db623b8a0c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:59.302339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60085-99f5-4100-9046-09db623b8a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:59.302711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60085-99f5-4100-9046-09db623b8a0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:59.316126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b60085-99f5-4100-9046-09db623b8a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d3b60085-99f5-4100-9046-09db623b8a0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:05:59.326982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:59.328172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:05:59.329214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:59.336117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 302/606] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:05:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:59.336117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b60085-99f5-4100-9046-09db623b8a0c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.347285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a4f5b-6e36-40a8-94ea-da24684e9ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:59.348438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a4f5b-6e36-40a8-94ea-da24684e9ee7 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:59.348866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a4f5b-6e36-40a8-94ea-da24684e9ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:59.349285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a4f5b-6e36-40a8-94ea-da24684e9ee7 None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/d7cebda9ae4e4a6b9a9bbface3e7b232/roles/d82425658c8545e6b546d601ef7b9b58` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:59.366078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a4f5b-6e36-40a8-94ea-da24684e9ee7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=d7cebda9ae4e4a6b9a9bbface3e7b232, role_id=d82425658c8545e6b546d601ef7b9b58)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:59.367743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a4f5b-6e36-40a8-94ea-da24684e9ee7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:59.414555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:59.445268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b60085-99f5-4100-9046-09db623b8a0c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:59.480864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 305/607] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:59 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/d7cebda9ae4e4a6b9a9bbface3e7b232/roles/d82425658c8545e6b546d601ef7b9b58 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:05:59.482296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a4f5b-6e36-40a8-94ea-da24684e9ee7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.484277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:59.485084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:59.485418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:59.485910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:59.509549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:59.514199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:59.519702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:05:59.550805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:59.551813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:59.552694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:05:59.552694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:05:59.552694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:05:59.552694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:05:59.552694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:05:59.552694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:05:59.552694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:05:59.552694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:05:59.555440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 303/608] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:59.560034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.695465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a4f5b-6e36-40a8-94ea-da24684e9ee7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:59.734634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:59.734996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5046dae-1923-4ce5-9f98-b9bc29b7655b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:59.735887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:59.736476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:59.736979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:59.737989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:59.738501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:59.740177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:59.750145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 306/609] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:05:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:59.756566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.773712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:59.781264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:59.782202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:59.782490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:59.784552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:59.784552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:05:59.784552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:59.785338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:59.811568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 304/610] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:05:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:05:59.815723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.875615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:05:59.906962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce165-d9c1-4beb-abdb-c5ab8e58a2a2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:05:59.942421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:05:59.943200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:05:59.943509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:05:59.943784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:05:59.955334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:05:59.956105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:05:59.974285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:05:59.975433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:05:59.976458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:05:59.976458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:05:59.976458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:05:59.976458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:05:59.982421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 307/611] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:05:59.983773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:05:59.996959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc5753-1da6-42eb-99e6-babf43a997b0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:00.034122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b6f29-db3c-451e-94e2-ded87ae898ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:00.035479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b6f29-db3c-451e-94e2-ded87ae898ec None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:00.035986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b6f29-db3c-451e-94e2-ded87ae898ec None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:00.036507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b6f29-db3c-451e-94e2-ded87ae898ec None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/cb1fa1e11b1a44aaa31cc19f7c199285/roles/d82425658c8545e6b546d601ef7b9b58` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:00.052950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b6f29-db3c-451e-94e2-ded87ae898ec None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=cb1fa1e11b1a44aaa31cc19f7c199285, role_id=d82425658c8545e6b546d601ef7b9b58)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:00.054720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b6f29-db3c-451e-94e2-ded87ae898ec None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:00.125386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd5f60-daba-45f5-b093-2bece58c9d0b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:00.151320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:00.162966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 305/612] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:05:59 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/cb1fa1e11b1a44aaa31cc19f7c199285/roles/d82425658c8545e6b546d601ef7b9b58 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:06:00.173065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:00.174344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:00.175165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:00.175722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:00.176751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:00.177361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:00.179232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:00.196286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 308/613] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:00.206288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b6f29-db3c-451e-94e2-ded87ae898ec None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:00.275590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:00.354663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b6f29-db3c-451e-94e2-ded87ae898ec None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:00.394610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faad6cd-d7d3-464c-a85e-14b2b2bfcab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:00.395451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad6cd-d7d3-464c-a85e-14b2b2bfcab4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:00.395785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad6cd-d7d3-464c-a85e-14b2b2bfcab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:00.396064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad6cd-d7d3-464c-a85e-14b2b2bfcab4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:00.403753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faad6cd-d7d3-464c-a85e-14b2b2bfcab4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:00.405225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faad6cd-d7d3-464c-a85e-14b2b2bfcab4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5faad6cd-d7d3-464c-a85e-14b2b2bfcab4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:00.423494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:00.424767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:00.425704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:00.427849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 306/614] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:00.431214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:00.517626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa3df9-7074-4cda-a807-ffb560799a2c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:00.518178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa3df9-7074-4cda-a807-ffb560799a2c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:00.518592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa3df9-7074-4cda-a807-ffb560799a2c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:00.520438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 307/615] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:00.525933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c4d6e-31e6-43ee-acea-189addcf48e1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:00.526884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c4d6e-31e6-43ee-acea-189addcf48e1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:00.527328 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c4d6e-31e6-43ee-acea-189addcf48e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:00.546910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-134c4d6e-31e6-43ee-acea-189addcf48e1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:00.548279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-134c4d6e-31e6-43ee-acea-189addcf48e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:00.570291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4261ed-7d76-4208-8628-a8d1fc1a2ddc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:00.626881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 308/616] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:06:00.651828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c4d6e-31e6-43ee-acea-189addcf48e1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:00.669397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:00.670080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:00.670409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:00.670694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:00.671574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:00.671883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:00.673317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:00.691715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:00.715850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 309/617] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 11 11:06:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:00.721880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:00.793069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c4d6e-31e6-43ee-acea-189addcf48e1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:00.814092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:00.847845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee13fdd-0535-4e43-8333-84803327aecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:00.849184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee13fdd-0535-4e43-8333-84803327aecf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:00.849704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee13fdd-0535-4e43-8333-84803327aecf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:00.850087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee13fdd-0535-4e43-8333-84803327aecf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:00.863620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee13fdd-0535-4e43-8333-84803327aecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2ee13fdd-0535-4e43-8333-84803327aecf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:00.875020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:00.876160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:00.877209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 309/618] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:00.881339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee13fdd-0535-4e43-8333-84803327aecf None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:00.945214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51cc98-4bd2-43e2-b373-d7239d88c71e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:00.980314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:00.980971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:00.981090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:00.981384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:00.981665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:00.984276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee13fdd-0535-4e43-8333-84803327aecf None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:00.992530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:00.994058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:01.020182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:01.021168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:01.022452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:01.022452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:01.022452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:01.022452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:01.022452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 310/619] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:01.026603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.027459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.027816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.028221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.029044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:01.029421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.030877 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.037712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.040904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 310/620] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:06:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:01.046621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.151389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e0d4b-3aad-4cef-8560-591621974c3b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.190044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:01.190966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246dd0d-291f-4204-a30b-9041748e19c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.192117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246dd0d-291f-4204-a30b-9041748e19c0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.192465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246dd0d-291f-4204-a30b-9041748e19c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.192812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246dd0d-291f-4204-a30b-9041748e19c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.197119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c29f0a-2773-4039-b2a8-14e188261787 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.204200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246dd0d-291f-4204-a30b-9041748e19c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.205719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246dd0d-291f-4204-a30b-9041748e19c0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8246dd0d-291f-4204-a30b-9041748e19c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:01.229124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:01.230778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:01.231819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:01.231819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:01.231819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:01.231819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:01.237898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 311/621] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:01.239650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2513a82-1188-4c18-95e0-0f4621a8d088 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.240053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2513a82-1188-4c18-95e0-0f4621a8d088 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.240509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2513a82-1188-4c18-95e0-0f4621a8d088 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.241018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84714d-1fab-4012-bbef-cd3ee623e6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.241744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84714d-1fab-4012-bbef-cd3ee623e6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.242093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84714d-1fab-4012-bbef-cd3ee623e6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.242262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 312/622] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:01.242551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84714d-1fab-4012-bbef-cd3ee623e6a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.242974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2513a82-1188-4c18-95e0-0f4621a8d088 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.243566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84714d-1fab-4012-bbef-cd3ee623e6a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:01.243967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84714d-1fab-4012-bbef-cd3ee623e6a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.245454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84714d-1fab-4012-bbef-cd3ee623e6a4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.256050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:01.257499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 311/623] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:01.258851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8faf14-e4b7-421c-964d-226011afce00 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.259240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8faf14-e4b7-421c-964d-226011afce00 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.259617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8faf14-e4b7-421c-964d-226011afce00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.283289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7a8faf14-e4b7-421c-964d-226011afce00 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:01.284555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7a8faf14-e4b7-421c-964d-226011afce00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:01.365816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2513a82-1188-4c18-95e0-0f4621a8d088 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.375379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 312/624] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:01.376309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8faf14-e4b7-421c-964d-226011afce00 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.414466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.415178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.415564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.415848 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.416646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:01.416975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.418351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.435551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 313/625] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:01.436418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.470898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:01.486970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8faf14-e4b7-421c-964d-226011afce00 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.509651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:01.524102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.525199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.525626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.526580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.533853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.535530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.548275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3151d-d01c-4eeb-a537-5d8d1cdbb5d2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:01.550306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:01.551410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:01.552456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:01.552456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:01.552456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:01.552456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:01.552456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:01.552456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:01.552456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:01.552456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:01.554562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 313/626] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:01.555529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.591610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5833b29-5c69-4c97-8236-2271b88a83fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.592426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5833b29-5c69-4c97-8236-2271b88a83fa None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.592720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5833b29-5c69-4c97-8236-2271b88a83fa None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.592997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5833b29-5c69-4c97-8236-2271b88a83fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.604587 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5833b29-5c69-4c97-8236-2271b88a83fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-b5833b29-5c69-4c97-8236-2271b88a83fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:01.616119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:01.617341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:01.618295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:01.622484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 314/627] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:01.623363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5833b29-5c69-4c97-8236-2271b88a83fa None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.666316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7d3e1-bdd7-4958-94bf-7a40e741036f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.706381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.707623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.707911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.708593 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.718768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.720898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.735958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5833b29-5c69-4c97-8236-2271b88a83fa None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:01.740941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:01.759134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 314/628] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.760881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:01.773772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:01.784540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.785905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.786644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.787071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.788216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:01.789009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.790814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.806779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 315/629] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:01.808006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.876290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52c172-151d-4f55-acd8-aa212287d0c5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.914663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.915377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.915670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.915962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.917281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:01.917535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.918991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:01.933037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 315/630] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:01.933907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:01.943438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15182888-ed4b-4917-9e82-523f60790aa8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:01.980231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:01.981211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:01.981332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:01.981533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:01.982478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:01.982798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:01.984278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:02.006652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 316/631] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:06:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:02.007869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:02.022052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:02.062558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:02.074082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a841a-09b4-44e3-b417-a253aea9e296 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:02.117472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b726c-e473-4ee9-ad7b-71e2d35c09eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:02.118755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b726c-e473-4ee9-ad7b-71e2d35c09eb None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.119292 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b726c-e473-4ee9-ad7b-71e2d35c09eb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.120312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b726c-e473-4ee9-ad7b-71e2d35c09eb None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/7556ce4bf09d4113aec2037dd438cecb/roles/d82425658c8545e6b546d601ef7b9b58` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.125704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c12f01-094b-4ee1-9b0c-54226df3e3f3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:02.136036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b726c-e473-4ee9-ad7b-71e2d35c09eb None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=7556ce4bf09d4113aec2037dd438cecb, role_id=d82425658c8545e6b546d601ef7b9b58)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:02.137856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b726c-e473-4ee9-ad7b-71e2d35c09eb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:02.162112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:02.162893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.163176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.163473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.174071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:02.175751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:02.192840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:02.193794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:02.194705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:02.194705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:02.194705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:02.194705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:02.194705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 317/632] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:02.195629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:02.211124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 316/633] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:06:01 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/7556ce4bf09d4113aec2037dd438cecb/roles/d82425658c8545e6b546d601ef7b9b58 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:06:02.211124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc7032-047b-4d8d-bfa3-ceda1e6cddcc None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.211124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc7032-047b-4d8d-bfa3-ceda1e6cddcc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.211124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc7032-047b-4d8d-bfa3-ceda1e6cddcc None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.211124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 317/634] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:02.211124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc7032-047b-4d8d-bfa3-ceda1e6cddcc None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:02.318991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:02.325392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4372a73-d543-4222-bc29-14361e948109 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:02.349976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc7032-047b-4d8d-bfa3-ceda1e6cddcc None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:02.365514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfab1d-ab8d-4833-8dac-8843e96b4761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:02.366182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfab1d-ab8d-4833-8dac-8843e96b4761 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.366471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfab1d-ab8d-4833-8dac-8843e96b4761 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.366751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfab1d-ab8d-4833-8dac-8843e96b4761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.374429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:02.400166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242eda9-7ef5-4157-a173-b2b7c5d9355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:02.401493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242eda9-7ef5-4157-a173-b2b7c5d9355c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.402419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242eda9-7ef5-4157-a173-b2b7c5d9355c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.402697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242eda9-7ef5-4157-a173-b2b7c5d9355c None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.403686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242eda9-7ef5-4157-a173-b2b7c5d9355c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:02.403996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242eda9-7ef5-4157-a173-b2b7c5d9355c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:02.405367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242eda9-7ef5-4157-a173-b2b7c5d9355c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:02.418910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 318/635] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:02.419368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd07cd0-201e-43da-8e90-74b00576f4f6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.419667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd07cd0-201e-43da-8e90-74b00576f4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.419944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd07cd0-201e-43da-8e90-74b00576f4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.438838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-edd07cd0-201e-43da-8e90-74b00576f4f6 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:02.439977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-edd07cd0-201e-43da-8e90-74b00576f4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:02.469251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfab1d-ab8d-4833-8dac-8843e96b4761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:02.471875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfab1d-ab8d-4833-8dac-8843e96b4761 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:02.517752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 319/636] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:02.518849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd07cd0-201e-43da-8e90-74b00576f4f6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:02.593397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:02.600116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 318/637] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:06:02.601426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8cd76-27a3-4960-9c70-b0868ae371b0 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.601719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8cd76-27a3-4960-9c70-b0868ae371b0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.606037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8cd76-27a3-4960-9c70-b0868ae371b0 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.606037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 319/638] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:02.606037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8cd76-27a3-4960-9c70-b0868ae371b0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:02.643421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd07cd0-201e-43da-8e90-74b00576f4f6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:02.650687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:02.678269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:02.679060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.679376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.679654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] PATH_INFO: `/v3/users/glance` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.686822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:02.688294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:02.702813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:02.703809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 320/639] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:02.704750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:02.716173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8cd76-27a3-4960-9c70-b0868ae371b0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:02.751890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f8654-bb0a-48a0-a6aa-7a0c25fd5c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:02.752686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f8654-bb0a-48a0-a6aa-7a0c25fd5c62 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.752966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f8654-bb0a-48a0-a6aa-7a0c25fd5c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.753243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f8654-bb0a-48a0-a6aa-7a0c25fd5c62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.765070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f8654-bb0a-48a0-a6aa-7a0c25fd5c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-786f8654-bb0a-48a0-a6aa-7a0c25fd5c62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:02.773219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:02.780685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 320/640] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:02.781554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cbf67-074c-416e-bd59-bb05c504266f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.782340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cbf67-074c-416e-bd59-bb05c504266f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.782340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cbf67-074c-416e-bd59-bb05c504266f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.793422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-586cbf67-074c-416e-bd59-bb05c504266f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:02.794517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-586cbf67-074c-416e-bd59-bb05c504266f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:02.830528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90aa8-d7c8-450e-9b5c-61e4b648afde None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:02.847238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:02.864811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 321/641] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:02.865727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cbf67-074c-416e-bd59-bb05c504266f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:02.878015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:02.879039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:02.879540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:02.879998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:02.893972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:02.895907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:02.914726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:02.915776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:02.916822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:02.916822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:02.916822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:02.916822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:02.920398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 321/642] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:02.921301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:02.942106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:02.984694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cbf67-074c-416e-bd59-bb05c504266f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.021875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.022548 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.022768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54c833-9200-430b-b76c-5a262f64e6c8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.022873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.023100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.023961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:03.024391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.025969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.048656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 322/643] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 11 11:06:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:03.048656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.066007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.066681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.066973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.067301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.068065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:03.068376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.069748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.080749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 322/644] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:06:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:03.081442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.108176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:03.163327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d094c-7afe-4867-9e09-64ed2a5a6509 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.165315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:03.184519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b3cf3-1434-4d67-b472-4c9afd5e81b7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.202126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc31e62-c690-4521-82cc-7c9bd43b99b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.202924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc31e62-c690-4521-82cc-7c9bd43b99b0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.203210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc31e62-c690-4521-82cc-7c9bd43b99b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.203516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc31e62-c690-4521-82cc-7c9bd43b99b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.214604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc31e62-c690-4521-82cc-7c9bd43b99b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:03.219429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.220086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.220366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.220639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.221350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:03.221656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2bc31e62-c690-4521-82cc-7c9bd43b99b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:03.222400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:03.223322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:03.229421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:03.230255 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.230345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 323/645] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:03.230345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc31e62-c690-4521-82cc-7c9bd43b99b0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.242665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 323/646] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:03.247709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.341870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc31e62-c690-4521-82cc-7c9bd43b99b0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.349100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:03.363480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d359b3-bddb-4822-8710-ae6c9c8901e8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.383951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce011611-4eb5-4fe6-879d-cda64d2ca17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.385000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011611-4eb5-4fe6-879d-cda64d2ca17f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.385335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011611-4eb5-4fe6-879d-cda64d2ca17f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.385614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011611-4eb5-4fe6-879d-cda64d2ca17f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.401924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.402714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.402994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.403279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.412418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:03.414871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.416363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:03.435138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:03.436102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:03.436995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:03.436995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:03.436995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:03.436995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:03.436995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 324/647] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:03.437289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.461539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce011611-4eb5-4fe6-879d-cda64d2ca17f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.462985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce011611-4eb5-4fe6-879d-cda64d2ca17f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.546872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57365494-61f3-44ab-b9d7-102d3a8ab66e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.569659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 324/648] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:06:03.569659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce011611-4eb5-4fe6-879d-cda64d2ca17f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.572179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:03.584720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.589468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.589468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.589468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.589468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:03.589468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.589468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.601923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 325/649] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:03.601923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.657794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:03.691035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce011611-4eb5-4fe6-879d-cda64d2ca17f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.705721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aace384-234e-4e84-81a8-3811fdcb43ae None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.727388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.728339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.728625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.728922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.736152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.737588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.741618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.742282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.742569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.742849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.743608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:03.743921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.745309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:03.751208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:03.752106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:03.752988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:03.752988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:03.752988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:03.752988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:03.752988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:03.752988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:03.752988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:03.752988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:03.755566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 325/650] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:03.756703 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.760324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 326/651] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:03.761402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.822149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:03.863125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403d5c6-f740-4b68-8571-cd2fa6ada5c2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.864987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859c200-f94e-4de7-a0c2-4462af241c6f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:03.879991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:03.898258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.899042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.899364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.899639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.901522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:03.902310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:03.902657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:03.902928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:03.911093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.912722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.913037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:03.914507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:03.931931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:03.933419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:03.934303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:03.934303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:03.934303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:03.934303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:03.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:03.935367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:03.937015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:03.937015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:03.937015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:03.937015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:03.937015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 327/652] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:03.937015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:03.938706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 326/653] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:03.939836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.043277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eac949-f22c-4703-985e-b8329acf030b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.043519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155978e-ff27-4f10-9ba8-0d15ae633e0b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.066709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:04.085369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.086169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.086587 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.086765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.091280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.091988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.092276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.092554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:04.093669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.094573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.095108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.096016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.112239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 328/654] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:04.112239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:04.112898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:04.118629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 327/655] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:04.120276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.147141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:04.220812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946116-197c-4eab-82f9-812196fb1e2d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.226401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3c236-1a9b-4bcb-93a1-7787b59e5565 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.257397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.258062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.258347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.258622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.259372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:04.259686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.260313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.260965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.261060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.261246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.261517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.262237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:04.262531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.263923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.273675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 328/656] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:04.274304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.285106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 329/657] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:04.285106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.312810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:04.373135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:04.383537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a523341-76fc-4b87-a5e2-c17b2c27800b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.387961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a2b5a-2147-4554-af40-994c131f2bfa None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.415970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.416642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.416922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.417204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.418014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=258ce5f4d6ff400199a80157cc34f5e5) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:04.418323 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.419767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.421974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.423049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.423366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.423640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.424752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:04.425058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.425673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 329/658] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/role_assignments?user.id=258ce5f4d6ff400199a80157cc34f5e5&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:04.426448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.426545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.454324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 330/659] 10.222.0.26 () {56 vars in 1137 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:04.455181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.538292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:04.542978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0d801-64ff-4db7-b1cf-50afc8125fcd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.576471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a0d6-b4b2-4b1a-88a3-5b71e86d2ef7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.581459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961e16e-5aff-45f7-8d14-f78fc52bcd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.582232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961e16e-5aff-45f7-8d14-f78fc52bcd7c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.582519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961e16e-5aff-45f7-8d14-f78fc52bcd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.582799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961e16e-5aff-45f7-8d14-f78fc52bcd7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.593282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961e16e-5aff-45f7-8d14-f78fc52bcd7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.594735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961e16e-5aff-45f7-8d14-f78fc52bcd7c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-b961e16e-5aff-45f7-8d14-f78fc52bcd7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:04.608904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:04.609857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:04.610765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 330/660] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:04.612225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52306433-3dbd-455d-b4a5-04a3658e0aeb None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.612512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52306433-3dbd-455d-b4a5-04a3658e0aeb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.613024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52306433-3dbd-455d-b4a5-04a3658e0aeb None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.614255 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 331/661] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:04.614989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52306433-3dbd-455d-b4a5-04a3658e0aeb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.626427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.627165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.627521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.627804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.639582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.640992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:04.659490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:04.660639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:04.661612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 331/662] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:04.662302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.665638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:04.733753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52306433-3dbd-455d-b4a5-04a3658e0aeb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.768270 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3bfdc-1f46-4035-a841-42bf2e79a4bf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:04.770460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02d73c-0721-4187-8539-e65bcad6d090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.771300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02d73c-0721-4187-8539-e65bcad6d090 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.771732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02d73c-0721-4187-8539-e65bcad6d090 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.772178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02d73c-0721-4187-8539-e65bcad6d090 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.801740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:04.802746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b3459-bde3-48d6-8c4e-9111f9fab019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:04.803440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3459-bde3-48d6-8c4e-9111f9fab019 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.803728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3459-bde3-48d6-8c4e-9111f9fab019 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.804006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b3459-bde3-48d6-8c4e-9111f9fab019 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.804801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b3459-bde3-48d6-8c4e-9111f9fab019 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:04.805119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b3459-bde3-48d6-8c4e-9111f9fab019 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.806566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b3459-bde3-48d6-8c4e-9111f9fab019 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.823854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 332/663] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:04.823854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5134-01f3-4ce1-8b1e-226ba3e05df9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:04.823854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5134-01f3-4ce1-8b1e-226ba3e05df9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:04.823854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5134-01f3-4ce1-8b1e-226ba3e05df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:04.838167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-597b5134-01f3-4ce1-8b1e-226ba3e05df9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:04.839489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-597b5134-01f3-4ce1-8b1e-226ba3e05df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:04.852746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02d73c-0721-4187-8539-e65bcad6d090 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:04.855268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02d73c-0721-4187-8539-e65bcad6d090 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:04.903434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:04.923274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 333/664] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:04.923274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5134-01f3-4ce1-8b1e-226ba3e05df9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:04.964312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 332/665] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:06:04.968843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02d73c-0721-4187-8539-e65bcad6d090 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.031132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5134-01f3-4ce1-8b1e-226ba3e05df9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.031381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:05.070359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.070359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.070359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.070359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.070777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:05.070777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:05.071996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:05.081584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02d73c-0721-4187-8539-e65bcad6d090 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.092576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 334/666] 10.222.0.26 () {56 vars in 1139 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:05.093394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.123680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.124345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.124632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.124940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.125739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=e939bc73c40140cb89517e918722d154) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:05.126051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:05.127723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:05.143639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 333/667] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:04 2022] GET /identity/v3/role_assignments?user.id=e939bc73c40140cb89517e918722d154&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:05.144470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.152961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:05.206575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92439b-fe6b-4fd2-a1c6-0af7b244eada None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.244942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12338-1375-4454-98fe-424edb3d3036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.245771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12338-1375-4454-98fe-424edb3d3036 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.246031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12338-1375-4454-98fe-424edb3d3036 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.246308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12338-1375-4454-98fe-424edb3d3036 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.258521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12338-1375-4454-98fe-424edb3d3036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-14a12338-1375-4454-98fe-424edb3d3036 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:05.266501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:05.267563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:05.268426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:05.272773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 335/668] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:05.273741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12338-1375-4454-98fe-424edb3d3036 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.294963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062a215-981f-4cf9-ad64-e9f4851ac6d6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.327376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:05.344430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.345237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.345517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.345787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.359001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:05.360639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:05.381097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:05.382048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:05.382984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:05.382984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:05.382984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:05.382984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:05.383690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 334/669] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:05.384479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12338-1375-4454-98fe-424edb3d3036 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.417139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:05.447422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a87b1-7bd6-44bd-a858-6162eaed6b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.447422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a87b1-7bd6-44bd-a858-6162eaed6b12 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.447422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a87b1-7bd6-44bd-a858-6162eaed6b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.447422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a87b1-7bd6-44bd-a858-6162eaed6b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.498266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a0353-f567-426d-90e2-1c1cd3f1e651 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.541066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.543226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.544027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.544509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.544702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a87b1-7bd6-44bd-a858-6162eaed6b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:05.545665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:05.546156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a87b1-7bd6-44bd-a858-6162eaed6b12 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:05.546378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:05.547963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:05.560448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 335/670] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:05.561865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.606630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:05.672531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 336/671] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:06:05.673452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a87b1-7bd6-44bd-a858-6162eaed6b12 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.678192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e3c68-0620-4498-95c8-4ab1f841492c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.688607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:05.717613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.718631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.719189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.719826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.721009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:05.721516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:05.723074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:05.740735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 336/672] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:05.741898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.779684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a87b1-7bd6-44bd-a858-6162eaed6b12 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.816660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.818073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.818073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.818073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.829100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:05.830651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:05.847001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:05.849978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:05.850866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:05.851889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:05.851889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:05.851889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:05.851889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:05.851889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 337/673] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:05.852565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.854907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7203c-1add-4c0e-a173-b880ca49f05e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.892098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.893321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.893804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.894277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:05.906234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:05.908032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:05.916193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:05.929560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:05.931438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:05.932903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:05.932903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:05.932903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:05.932903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:05.934399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 337/674] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:05.935582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:05.960364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0ee00-2d56-4edf-8c9d-0f5dae589627 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:05.997697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:05.998291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:05.998574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:05.998844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.006141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.007617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:06.022082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:06.023027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:06.023980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:06.023980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:06.023980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:06.023980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:06.023980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:06.023980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:06.023980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:06.023980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:06.027362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 338/675] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:06.028357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.048116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42353c27-0ffc-46c2-b0fa-7ecaa9b6e9e3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.089194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.090189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.090682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.091154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.092368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:06.092883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.094540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.107023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 338/676] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:06.108155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.121286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:06.134641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56694a-5e26-463c-930b-38f8b4721644 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.159866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:06.170730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.171657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.171917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.172195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.172918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:06.173250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.174875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.191790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 339/677] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:06.192664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.223203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76565621-d214-4c45-b9a5-7d6925586ae8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.260662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.261788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.262388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.262862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.263896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:06.264404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.266245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.279507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 339/678] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:06.280570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.302877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd0526-a9b3-4fd6-bf18-8b8f9d1a04f8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.339688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.340765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.341130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.341691 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.349313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.350743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.356629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:06.366753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:06.373748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 340/679] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:06.374864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.393665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04257-7d79-43ad-b5cf-7880a8c493ea None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.430168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:06.439606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.440672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.440955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.441231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.450371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.451837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:06.469452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:06.470449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:06.471375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:06.471375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:06.471375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:06.471375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:06.471375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:06.471375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:06.471375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:06.471375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:06.475321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 340/680] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:06.477140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.497337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c02519-9b2c-46d3-b999-b3329657d1f2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.534785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.535478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.535762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.536036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.536869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=d7cebda9ae4e4a6b9a9bbface3e7b232) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:06.537175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.539002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.555380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 341/681] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/role_assignments?user.id=d7cebda9ae4e4a6b9a9bbface3e7b232&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:06.555380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.612156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f609b-7e12-482a-91fc-6bcfec4f9403 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.643955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:06.654155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2ecab-06fc-417e-84b5-9920b5c2c686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.654838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2ecab-06fc-417e-84b5-9920b5c2c686 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.655120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2ecab-06fc-417e-84b5-9920b5c2c686 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.655446 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2ecab-06fc-417e-84b5-9920b5c2c686 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.656294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2ecab-06fc-417e-84b5-9920b5c2c686 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:06.656604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2ecab-06fc-417e-84b5-9920b5c2c686 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.658928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2ecab-06fc-417e-84b5-9920b5c2c686 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.686497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 341/682] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:06.687956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd7d0-7fec-4eda-b3c4-ecc5ecd6f062 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.688245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd7d0-7fec-4eda-b3c4-ecc5ecd6f062 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.688521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffd7d0-7fec-4eda-b3c4-ecc5ecd6f062 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.690146 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 342/683] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:06.696117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffd7d0-7fec-4eda-b3c4-ecc5ecd6f062 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.698305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe1d7d-5294-49fb-ae1c-e211306d506d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.710391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:06.742035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea804e-ccc6-43ff-807a-4bdf0afc6428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.742701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea804e-ccc6-43ff-807a-4bdf0afc6428 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.742983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea804e-ccc6-43ff-807a-4bdf0afc6428 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.743276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea804e-ccc6-43ff-807a-4bdf0afc6428 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.744272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea804e-ccc6-43ff-807a-4bdf0afc6428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:06.745342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea804e-ccc6-43ff-807a-4bdf0afc6428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.746852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea804e-ccc6-43ff-807a-4bdf0afc6428 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.761878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 342/684] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:06.763102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a8ea-c68f-405e-88d7-1611ac3d849a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.763488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a8ea-c68f-405e-88d7-1611ac3d849a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.763765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a8ea-c68f-405e-88d7-1611ac3d849a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.781916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3cd0a8ea-c68f-405e-88d7-1611ac3d849a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:06.782649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3cd0a8ea-c68f-405e-88d7-1611ac3d849a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:06.836124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffd7d0-7fec-4eda-b3c4-ecc5ecd6f062 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.859077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 343/685] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:06.860077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a8ea-c68f-405e-88d7-1611ac3d849a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:06.890862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521477ae-e9a7-43a9-a32a-803dbcc4e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:06.891833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521477ae-e9a7-43a9-a32a-803dbcc4e539 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:06.892120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521477ae-e9a7-43a9-a32a-803dbcc4e539 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:06.892389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521477ae-e9a7-43a9-a32a-803dbcc4e539 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:06.970693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521477ae-e9a7-43a9-a32a-803dbcc4e539 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:06.971785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:06.972849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521477ae-e9a7-43a9-a32a-803dbcc4e539 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:06.976543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a8ea-c68f-405e-88d7-1611ac3d849a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:06.983687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:07.012594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.013984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.014450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.015039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.016054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=cb1fa1e11b1a44aaa31cc19f7c199285) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:07.016580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.018222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.025480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 344/686] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/role_assignments?user.id=cb1fa1e11b1a44aaa31cc19f7c199285&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:07.027222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.104515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 343/687] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:06:07.109959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521477ae-e9a7-43a9-a32a-803dbcc4e539 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.137330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d6a59-211d-48a6-8ed9-9a0b07a1ffbb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.174997 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fde23e-0ab2-4285-84e6-d6203ac8633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.176261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fde23e-0ab2-4285-84e6-d6203ac8633c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.176747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fde23e-0ab2-4285-84e6-d6203ac8633c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.177216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fde23e-0ab2-4285-84e6-d6203ac8633c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.196217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fde23e-0ab2-4285-84e6-d6203ac8633c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-51fde23e-0ab2-4285-84e6-d6203ac8633c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:07.205871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:07.206865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:07.207763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:07.211492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 345/688] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:07.213070 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fde23e-0ab2-4285-84e6-d6203ac8633c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.238546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:07.258049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:07.269076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521477ae-e9a7-43a9-a32a-803dbcc4e539 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.308719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.309520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.309805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.310080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.325679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.327450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.328851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fde23e-0ab2-4285-84e6-d6203ac8633c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:07.351108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:07.352127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:07.353052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:07.353052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:07.353052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:07.353052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:07.353052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 344/689] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:07.359217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.372475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca98726-4649-4bba-9bdc-501c7a5f2d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.373149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98726-4649-4bba-9bdc-501c7a5f2d27 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.373438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98726-4649-4bba-9bdc-501c7a5f2d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.373707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98726-4649-4bba-9bdc-501c7a5f2d27 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.374494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca98726-4649-4bba-9bdc-501c7a5f2d27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:07.374797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca98726-4649-4bba-9bdc-501c7a5f2d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.376199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca98726-4649-4bba-9bdc-501c7a5f2d27 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.387119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 346/690] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:07.388757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5e65d-9c79-4e82-970c-ce582c6eca51 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.389180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5e65d-9c79-4e82-970c-ce582c6eca51 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.389583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5e65d-9c79-4e82-970c-ce582c6eca51 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.391745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 347/691] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:07.395841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5e65d-9c79-4e82-970c-ce582c6eca51 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.489501 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb820e-df93-4c99-9d00-78e40fe14da0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.504413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:07.521622 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5e65d-9c79-4e82-970c-ce582c6eca51 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.538990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645b524-55b1-4071-ad09-34beac78cae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.540034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645b524-55b1-4071-ad09-34beac78cae6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.540463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645b524-55b1-4071-ad09-34beac78cae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.540965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645b524-55b1-4071-ad09-34beac78cae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.541952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645b524-55b1-4071-ad09-34beac78cae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:07.542491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645b524-55b1-4071-ad09-34beac78cae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.544108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645b524-55b1-4071-ad09-34beac78cae6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.553780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:07.557725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 345/692] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:07.559299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c693f-c276-4aed-b4fd-6f828b1cc307 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.559784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c693f-c276-4aed-b4fd-6f828b1cc307 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.560223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c693f-c276-4aed-b4fd-6f828b1cc307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.565121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.565265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.565572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.566041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.578288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-463c693f-c276-4aed-b4fd-6f828b1cc307 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:07.579721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-463c693f-c276-4aed-b4fd-6f828b1cc307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:07.583977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.586957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:07.611152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:07.612193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:07.613096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:07.613096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:07.613096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:07.613096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:07.613320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 348/693] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:07.615290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.658670 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 346/694] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:07.659887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c693f-c276-4aed-b4fd-6f828b1cc307 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.745694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e96dde-daae-4a1a-a105-a5552a3852cc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.770146 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c693f-c276-4aed-b4fd-6f828b1cc307 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.781362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.782160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.782449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.782732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.785535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:07.790592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.792089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.803955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:07.807106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:07.808012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:07.809473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 349/695] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:07.809964 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.814661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.815640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.816121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.816605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.817521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:07.818025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.819738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.832506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 347/696] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:07.833694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:07.915048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf2af3-f9ec-411c-8694-01122b599693 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.939196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bcf40-2f3e-4f53-b62f-87b65720f7d4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:07.953416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.954131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.954417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.954696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.966105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.967588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.979687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:07.980716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:07.981241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:07.981725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:07.982698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:07.983201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:07.984882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:07.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:07.986848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:07.987821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:07.987821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:07.987821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:07.987821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:07.991967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 350/697] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:07.992811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:08.002549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 348/698] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:08.003859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:08.044826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:08.056329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:08.107061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a558f7-9ae4-4cc0-9789-9458908bc9f2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:08.107502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87edafee-96fb-4f9a-9884-fdea1e6702ed None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:08.142828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:08.143696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:08.144139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:08.144482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:08.144771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:08.144987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:08.145085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:08.145597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:08.146554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:08.147036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:08.148704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:08.152863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:08.154314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:08.161844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 349/699] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:08.162867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:08.173316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:08.174538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 351/700] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:06:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:08.176078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:08.268693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9a5112-7714-4cdb-a4fc-61fdd902dcf2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:08.278698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916abee-09f7-4df1-99c0-d400d19796d6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:08.280635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:08.289228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:08.305487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:08.306429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:08.306931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:08.307420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:08.308423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=7556ce4bf09d4113aec2037dd438cecb) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:08.308915 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:08.310524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:08.315947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c3841-8c45-402e-b6ac-e7c05239664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:08.316619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c3841-8c45-402e-b6ac-e7c05239664d None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:08.316967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c3841-8c45-402e-b6ac-e7c05239664d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:08.317176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c3841-8c45-402e-b6ac-e7c05239664d None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:08.317888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c3841-8c45-402e-b6ac-e7c05239664d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:08.319437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c3841-8c45-402e-b6ac-e7c05239664d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:08.319898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 350/701] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:08 2022] GET /identity/v3/role_assignments?user.id=7556ce4bf09d4113aec2037dd438cecb&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:08.321069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:08.430693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 352/702] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:06:08 2022] POST /identity/v3/users => generated 254 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:08.441147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4455c2f-bd10-49bd-a156-5d0a6b76e32b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:08.482034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:08.483032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:08.483525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:08.483978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:08.485060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:08.485559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:08.487217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:08.521881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 351/703] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:06:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:08.527942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c3841-8c45-402e-b6ac-e7c05239664d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:08.668864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c3841-8c45-402e-b6ac-e7c05239664d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:08.703858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc8d6f5-ef1a-4d69-b8ba-90c79988cbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:08.704440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8d6f5-ef1a-4d69-b8ba-90c79988cbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:08.704724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8d6f5-ef1a-4d69-b8ba-90c79988cbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:08.705002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8d6f5-ef1a-4d69-b8ba-90c79988cbe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:08.741541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:08.779399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8d6f5-ef1a-4d69-b8ba-90c79988cbe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:08.780927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc8d6f5-ef1a-4d69-b8ba-90c79988cbe1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:08.909325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 353/704] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:06:08.914825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:08.934624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31a090-a84f-498a-9644-e4e4af2987d2 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:08.934954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31a090-a84f-498a-9644-e4e4af2987d2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:08.935206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31a090-a84f-498a-9644-e4e4af2987d2 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:08.937165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 354/705] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:08.942738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35113a5-5583-4ef0-96d7-7bc706605e61 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:08.943151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35113a5-5583-4ef0-96d7-7bc706605e61 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:08.943599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35113a5-5583-4ef0-96d7-7bc706605e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:08.964471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a35113a5-5583-4ef0-96d7-7bc706605e61 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:08.965763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a35113a5-5583-4ef0-96d7-7bc706605e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:08.972321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:09.047277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:09.047545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 355/706] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:09.082887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35113a5-5583-4ef0-96d7-7bc706605e61 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:09.137857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796a353-dbab-470a-bd61-fc694b3f697f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:09.208930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:09.209739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.210023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.210296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.221541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:09.222928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:09.240684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:09.241630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:09.242543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:09.242543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:09.242543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:09.242543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:09.242781 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35113a5-5583-4ef0-96d7-7bc706605e61 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:09.242858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 352/707] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:09.247494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:09.304699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:09.305519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.305806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.306115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.307018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:09.308424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:09.331987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:09.337832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:09.338893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:09.339803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 356/708] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 11 11:06:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:09.346912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:09.371512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685ff52-273b-4715-8c63-00c56f72ec3a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:09.445998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:09.447882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585d389-c731-48c6-bb6f-f28924230b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:09.448551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d389-c731-48c6-bb6f-f28924230b31 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.448835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d389-c731-48c6-bb6f-f28924230b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.449108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585d389-c731-48c6-bb6f-f28924230b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.449834 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d389-c731-48c6-bb6f-f28924230b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:09.450140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d389-c731-48c6-bb6f-f28924230b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:09.451533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585d389-c731-48c6-bb6f-f28924230b31 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:09.464594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 353/709] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:09.465924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc45b6c-27da-4c32-b1d2-e0bcf9f62a75 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.466218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc45b6c-27da-4c32-b1d2-e0bcf9f62a75 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.466497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc45b6c-27da-4c32-b1d2-e0bcf9f62a75 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.473044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 354/710] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:09.474122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc45b6c-27da-4c32-b1d2-e0bcf9f62a75 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:09.502330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd907b65-aa20-4e3c-968f-127a721c2bcf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:09.547023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1464f-7f83-4f19-8626-46004a4ee71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:09.547717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1464f-7f83-4f19-8626-46004a4ee71c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.547999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1464f-7f83-4f19-8626-46004a4ee71c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.548275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1464f-7f83-4f19-8626-46004a4ee71c None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.548984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1464f-7f83-4f19-8626-46004a4ee71c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:09.549545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1464f-7f83-4f19-8626-46004a4ee71c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:09.551718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1464f-7f83-4f19-8626-46004a4ee71c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:09.557775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 357/711] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 11 11:06:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:09.558763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b50c42-27d2-44ed-9c15-cc70e448bb7b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.559055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b50c42-27d2-44ed-9c15-cc70e448bb7b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.559355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b50c42-27d2-44ed-9c15-cc70e448bb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.579321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-80b50c42-27d2-44ed-9c15-cc70e448bb7b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:09.580495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-80b50c42-27d2-44ed-9c15-cc70e448bb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:09.601424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc45b6c-27da-4c32-b1d2-e0bcf9f62a75 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:09.642281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:09.643374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.643871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.644343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.651318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:09.652973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:09.657711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 358/712] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:09.658631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b50c42-27d2-44ed-9c15-cc70e448bb7b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:09.668360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:09.669301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:09.670220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:09.670220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:09.670220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:09.670220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:09.670220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:09.670220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:09.670220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:09.670220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:09.672622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 355/713] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:09.680268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:09.686570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:09.730930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:09.783125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b50c42-27d2-44ed-9c15-cc70e448bb7b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:09.810529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ef91b-5523-4483-9b06-356462c745c4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:09.821559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:09.822236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.823036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.823371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.824098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:09.824416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:09.825801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:09.839570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 359/714] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 11 11:06:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:09.840897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:09.856450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:09.857595 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:09.858179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:09.858666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:09.859759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:09.860293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:09.861991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:09.877178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 356/715] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:09.884388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:10.012932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:10.047868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7df4a4-63bd-43f1-8ac5-ed2e58d382bf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:10.078028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:10.078438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c471c82-4633-4e6a-be18-3a84abafc448 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:10.101286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:10.102889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:10.103445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:10.104634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:10.105446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:10.107895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:10.119220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:10.120603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:10.120987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:10.121415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:10.122401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=258ce5f4d6ff400199a80157cc34f5e5) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:10.122816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:10.123548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:10.124741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:10.125712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:10.126457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:10.128633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 360/716] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:09 2022] GET /identity/v3/services/network => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:10.135330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 357/717] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:09 2022] GET /identity/v3/role_assignments?user.id=258ce5f4d6ff400199a80157cc34f5e5&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=007af6bd1b734517b934b4cac45faf2c => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:10.136564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:10.313000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4191a-a185-4d92-931c-29181f45e8b3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:10.357316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71442c-6a91-4931-80f0-3d7d941ae5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:10.357981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71442c-6a91-4931-80f0-3d7d941ae5ae None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:10.358261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71442c-6a91-4931-80f0-3d7d941ae5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:10.358535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71442c-6a91-4931-80f0-3d7d941ae5ae None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:10.359246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71442c-6a91-4931-80f0-3d7d941ae5ae None admin] RBAC: Adding query filter params (name=network) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:10.359573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71442c-6a91-4931-80f0-3d7d941ae5ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:10.360926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71442c-6a91-4931-80f0-3d7d941ae5ae None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:10.371841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 361/718] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:10.377038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:10.544934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee56c6-1279-4900-93bb-0353d3fe58c6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:10.573544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:10.586649 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049ab09-dc4c-4001-bdba-9e2bbc597609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:10.587617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ab09-dc4c-4001-bdba-9e2bbc597609 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:10.588378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ab09-dc4c-4001-bdba-9e2bbc597609 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:10.588705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049ab09-dc4c-4001-bdba-9e2bbc597609 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:10.589605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ab09-dc4c-4001-bdba-9e2bbc597609 None admin] RBAC: Adding query filter params (type=network) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:10.590099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ab09-dc4c-4001-bdba-9e2bbc597609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:10.591692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049ab09-dc4c-4001-bdba-9e2bbc597609 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:10.603283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 358/719] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:10.952470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331b5fb-9116-4029-a444-a0b4bfe0ae63 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:10.953016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331b5fb-9116-4029-a444-a0b4bfe0ae63 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:10.953016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331b5fb-9116-4029-a444-a0b4bfe0ae63 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:10.954663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 362/720] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:10.959437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6072-f7b2-4bcf-ba3c-ceea417ec6cd None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:10.959731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6072-f7b2-4bcf-ba3c-ceea417ec6cd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:10.960012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa6072-f7b2-4bcf-ba3c-ceea417ec6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:10.979129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dcfa6072-f7b2-4bcf-ba3c-ceea417ec6cd None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:10.980354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dcfa6072-f7b2-4bcf-ba3c-ceea417ec6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:11.075274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 359/721] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:06:11.108040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331b5fb-9116-4029-a444-a0b4bfe0ae63 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:11.160931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:11.181040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7393f9-930a-4485-a51f-ebfd20a15089 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:11.182191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7393f9-930a-4485-a51f-ebfd20a15089 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:11.182696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7393f9-930a-4485-a51f-ebfd20a15089 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:11.184594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 360/722] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:11.189847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134ae0a-d884-4f0a-bb28-034ec68c2a2d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:11.190397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134ae0a-d884-4f0a-bb28-034ec68c2a2d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:11.190911 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134ae0a-d884-4f0a-bb28-034ec68c2a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:11.208067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e134ae0a-d884-4f0a-bb28-034ec68c2a2d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:11.209550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e134ae0a-d884-4f0a-bb28-034ec68c2a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:11.236992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331b5fb-9116-4029-a444-a0b4bfe0ae63 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:11.282502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59042604-3a0d-46c4-997a-bf01398e3ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:11.283769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59042604-3a0d-46c4-997a-bf01398e3ddd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:11.284281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59042604-3a0d-46c4-997a-bf01398e3ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:11.284812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59042604-3a0d-46c4-997a-bf01398e3ddd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:11.287847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 361/723] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 11:06:11.308120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59042604-3a0d-46c4-997a-bf01398e3ddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-59042604-3a0d-46c4-997a-bf01398e3ddd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:11.316069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:11.317050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:11.317908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:11.327189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134ae0a-d884-4f0a-bb28-034ec68c2a2d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:11.327533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 363/724] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:11.333914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59042604-3a0d-46c4-997a-bf01398e3ddd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:11.419635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:11.495100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59042604-3a0d-46c4-997a-bf01398e3ddd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:11.509666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:11.512442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134ae0a-d884-4f0a-bb28-034ec68c2a2d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:11.531599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412eb1bd-076c-41b4-bf62-9ddb5d54ee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:11.532602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eb1bd-076c-41b4-bf62-9ddb5d54ee18 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:11.532891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eb1bd-076c-41b4-bf62-9ddb5d54ee18 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:11.533163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412eb1bd-076c-41b4-bf62-9ddb5d54ee18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:11.545388 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412eb1bd-076c-41b4-bf62-9ddb5d54ee18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-412eb1bd-076c-41b4-bf62-9ddb5d54ee18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:11.553518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:11.554530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:11.555512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 364/725] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:11.563031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412eb1bd-076c-41b4-bf62-9ddb5d54ee18 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:11.636378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:11.637554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:11.638071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:11.638583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:11.640058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:11.640383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:11.641797 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:11.662659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 362/726] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:11.676856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412eb1bd-076c-41b4-bf62-9ddb5d54ee18 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:11.680999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:11.716473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:11.717635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:11.717635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:11.717635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:11.718670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:11.718670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:11.720137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:11.729778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 365/727] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:06:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:11.735778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:11.784493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:11.955674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679e2fc-2af5-4f5d-97fa-e7c35b2e37af None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:11.997266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:11.998441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:11.998929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:11.999513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.009457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40539587-bb9a-4f7a-9c5f-512e257d7cb3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:12.012570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:12.014241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:12.035139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:12.036105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:12.037007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:12.037007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:12.037007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:12.037007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:12.043772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:12.044688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 366/728] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:12.046302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.088034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:12.088854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.089140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.089419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.107768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:12.109404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:12.132991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:12.138928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:12.139866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:12.140778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:12.140778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:12.140778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:12.140778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:12.148637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 363/729] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:12.154487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.158685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea505991-c27d-4387-b3e1-ee46bca5f01e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:12.207210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:12.208275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.208948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.209923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.211036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:12.211759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:12.214022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:12.245908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 367/730] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:12.247065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.342118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b3981-bf3a-4420-8b64-9bdd4fd20999 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:12.367265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4e664-0980-4e34-8baf-52ace0c6a867 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:12.376076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e51b7a-e5ba-4b97-ac5a-6abd450bc159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:12.376738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e51b7a-e5ba-4b97-ac5a-6abd450bc159 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.377020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e51b7a-e5ba-4b97-ac5a-6abd450bc159 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.377292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e51b7a-e5ba-4b97-ac5a-6abd450bc159 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.378199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e51b7a-e5ba-4b97-ac5a-6abd450bc159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:12.378311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e51b7a-e5ba-4b97-ac5a-6abd450bc159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:12.379909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e51b7a-e5ba-4b97-ac5a-6abd450bc159 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:12.393048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 364/731] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:12.394403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd267e7c-3af8-401b-b27f-7b177975dc96 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.394694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd267e7c-3af8-401b-b27f-7b177975dc96 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.394971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd267e7c-3af8-401b-b27f-7b177975dc96 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.396486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 365/732] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:12.398205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd267e7c-3af8-401b-b27f-7b177975dc96 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.406858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bede5-6d34-4e35-83c8-e9d8d009cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:12.408042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bede5-6d34-4e35-83c8-e9d8d009cae1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.408632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bede5-6d34-4e35-83c8-e9d8d009cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.409128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bede5-6d34-4e35-83c8-e9d8d009cae1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.418199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bede5-6d34-4e35-83c8-e9d8d009cae1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:12.419989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bede5-6d34-4e35-83c8-e9d8d009cae1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:12.422101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:12.426846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-165bede5-6d34-4e35-83c8-e9d8d009cae1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:12.438332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:12.439324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:12.440288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:12.442918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 368/733] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:12.448738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949debd7-d756-45ef-bfd7-7b7358c7961f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.449264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949debd7-d756-45ef-bfd7-7b7358c7961f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.449761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949debd7-d756-45ef-bfd7-7b7358c7961f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.468725 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-949debd7-d756-45ef-bfd7-7b7358c7961f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:12.470335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-949debd7-d756-45ef-bfd7-7b7358c7961f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:12.505001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd267e7c-3af8-401b-b27f-7b177975dc96 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:12.541445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e77a5-6396-4efa-93d4-526f09b3e0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:12.542238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e77a5-6396-4efa-93d4-526f09b3e0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.542521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e77a5-6396-4efa-93d4-526f09b3e0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.542804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e77a5-6396-4efa-93d4-526f09b3e0a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.545013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 369/734] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:12.546228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949debd7-d756-45ef-bfd7-7b7358c7961f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.550702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e77a5-6396-4efa-93d4-526f09b3e0a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:12.552162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e77a5-6396-4efa-93d4-526f09b3e0a5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-ca2e77a5-6396-4efa-93d4-526f09b3e0a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:12.568085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:12.569180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:12.570138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:12.573845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 366/735] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:12.575384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3b87d-1c71-428a-af2a-710ae9b44a7e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.575853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3b87d-1c71-428a-af2a-710ae9b44a7e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.576052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3b87d-1c71-428a-af2a-710ae9b44a7e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.577449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 367/736] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:12.578191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3b87d-1c71-428a-af2a-710ae9b44a7e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.664269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949debd7-d756-45ef-bfd7-7b7358c7961f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:12.669016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:12.685481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3b87d-1c71-428a-af2a-710ae9b44a7e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:12.693355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:12.701165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:12.701838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.702127 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.702435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.703299 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:12.703662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:12.705100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:12.722670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 370/737] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:12.723521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.727922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc264621-8278-4a19-b473-5d8f387cb639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:12.729061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc264621-8278-4a19-b473-5d8f387cb639 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.729534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc264621-8278-4a19-b473-5d8f387cb639 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.729813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc264621-8278-4a19-b473-5d8f387cb639 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.740760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc264621-8278-4a19-b473-5d8f387cb639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-bc264621-8278-4a19-b473-5d8f387cb639 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:12.748824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:12.749836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 368/738] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:12.756135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa91d-bc40-45d5-a960-089fd565b5b9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.756920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa91d-bc40-45d5-a960-089fd565b5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.756920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbfa91d-bc40-45d5-a960-089fd565b5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.772658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bdbfa91d-bc40-45d5-a960-089fd565b5b9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:12.774056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bdbfa91d-bc40-45d5-a960-089fd565b5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:12.838767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33e5b4-dab7-45ba-9806-8350af371acc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:12.845109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 369/739] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:12.856496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa91d-bc40-45d5-a960-089fd565b5b9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.874857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:12.875575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:12.875857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:12.876202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:12.876959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:12.877265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:12.878680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:12.898185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 371/740] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:12.899016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:12.947037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:12.949175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:12.970236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbfa91d-bc40-45d5-a960-089fd565b5b9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.002500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc559728-c6d0-4979-b8f5-31ecbcd9ebba None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.005968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1ad98-79ec-4ce8-b46a-f1bb51fae58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.006651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ad98-79ec-4ce8-b46a-f1bb51fae58b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.006913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ad98-79ec-4ce8-b46a-f1bb51fae58b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.007192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ad98-79ec-4ce8-b46a-f1bb51fae58b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.037463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.038129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.038431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.038687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.039452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:13.039758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.041634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.052718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 372/741] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:13.052718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.080881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1ad98-79ec-4ce8-b46a-f1bb51fae58b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.082333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1ad98-79ec-4ce8-b46a-f1bb51fae58b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.148976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f895c-e858-4b32-9013-dbba40549a37 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.155623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:13.157897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:13.180783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 370/742] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:06:13.181642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1ad98-79ec-4ce8-b46a-f1bb51fae58b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.182836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a06634-04bb-4d53-bcf9-8347edf4a130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.183538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a06634-04bb-4d53-bcf9-8347edf4a130 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.183828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a06634-04bb-4d53-bcf9-8347edf4a130 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.184107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a06634-04bb-4d53-bcf9-8347edf4a130 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.184830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a06634-04bb-4d53-bcf9-8347edf4a130 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.186516 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a06634-04bb-4d53-bcf9-8347edf4a130 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.261959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 373/743] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:06:13 2022] POST /identity/v3/services => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:13.263293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50accb65-a81c-4a86-a8ff-2144a12cdc97 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.263666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50accb65-a81c-4a86-a8ff-2144a12cdc97 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.264023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50accb65-a81c-4a86-a8ff-2144a12cdc97 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.265410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 374/744] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:13.265996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50accb65-a81c-4a86-a8ff-2144a12cdc97 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.285960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1ad98-79ec-4ce8-b46a-f1bb51fae58b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.339982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68fec8-fa13-4f49-afce-7f81a27928e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.340918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68fec8-fa13-4f49-afce-7f81a27928e1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.341506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68fec8-fa13-4f49-afce-7f81a27928e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.341937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68fec8-fa13-4f49-afce-7f81a27928e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.381292 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50accb65-a81c-4a86-a8ff-2144a12cdc97 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.391621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:13.418925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.420881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.420881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.420881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.428805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68fec8-fa13-4f49-afce-7f81a27928e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.430285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68fec8-fa13-4f49-afce-7f81a27928e1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.435437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.436992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:13.459692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:13.460642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:13.461539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:13.461539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:13.461539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:13.461539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:13.461539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 375/745] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:13.461866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.482546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:13.544321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 371/746] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:13 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 11:06:13.545356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4497de-4dfd-4074-833a-ae5d6e14a2ec None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.545647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4497de-4dfd-4074-833a-ae5d6e14a2ec None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.545926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4497de-4dfd-4074-833a-ae5d6e14a2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.561970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0e4497de-4dfd-4074-833a-ae5d6e14a2ec None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:13.563161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0e4497de-4dfd-4074-833a-ae5d6e14a2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:13.576214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3911-7602-43ee-8181-a69ff7af112d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.614655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.615759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.616046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.616318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.627863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.629281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.631440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 372/747] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 11:06:13.640299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4497de-4dfd-4074-833a-ae5d6e14a2ec None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.643005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:13.647303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:13.648196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:13.649105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:13.649105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:13.649105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:13.649105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:13.653423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 376/748] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:13.654301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.722666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:13.758035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4497de-4dfd-4074-833a-ae5d6e14a2ec None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.765031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078b40f-340c-472d-a1e6-9158d84c5737 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.794602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.795165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.795482 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.795762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.796481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:13.796792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.798279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.802179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.803210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.803798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.804088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.814553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 373/749] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:13.815424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.820234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.821840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:13.841062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:13.842037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:13.842970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:13.842970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:13.842970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:13.842970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:13.846788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 377/750] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:13.847468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.882589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:13.926096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7623e7-9370-45f7-9efa-4104ba36dd3f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.956423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482690f-1553-4e37-bb21-c49cd1a78aa7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:13.960613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.961275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.961559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.961836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:13.962552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:13.962900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:13.964293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:13.977109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 374/751] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:13.977941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:13.985059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:13.997644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf849d1-94aa-45cf-b6a0-bf1d3fbc7c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:13.998302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf849d1-94aa-45cf-b6a0-bf1d3fbc7c94 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:13.998590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf849d1-94aa-45cf-b6a0-bf1d3fbc7c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:13.998865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf849d1-94aa-45cf-b6a0-bf1d3fbc7c94 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.004033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf849d1-94aa-45cf-b6a0-bf1d3fbc7c94 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.004033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf849d1-94aa-45cf-b6a0-bf1d3fbc7c94 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.078894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bea408-6bd1-42d3-ad28-e3ec517229c2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.089072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 378/752] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:06:13 2022] POST /identity/v3/services => generated 240 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:14.100408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf849d1-94aa-45cf-b6a0-bf1d3fbc7c94 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.105597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:14.124718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.125529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.125814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.126176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.134826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.136736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:14.154949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:14.156057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:14.157020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:14.161285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 375/753] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:06:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:14.162421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.246874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf849d1-94aa-45cf-b6a0-bf1d3fbc7c94 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.277253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e790c4d-8403-4532-a505-eb4419f67b00 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.291758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.293155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.293588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.293876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.294616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:14.294931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.296783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.313970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 379/754] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:14.313970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.326701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.328139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.328430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.328711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.337079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.338677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:14.352631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:14.354095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:14.355309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 376/755] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:14.359903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.372799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:14.403862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:14.454725 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291d475-6114-4098-9423-3fb6c1bac0b1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.469798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec392c4a-0a51-4bea-9fc6-1ad50b987635 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.502238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.503271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.504050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.504489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.506225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:14.506689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.508372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.510371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.511393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.512837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.519394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.521027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.529648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 380/756] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:14.530807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:14.541426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:14.542527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:14.543657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:14.543657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:14.543657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:14.543657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:14.543657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:14.543657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:14.543657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:14.543657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:14.550296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 377/757] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:14.550296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.642173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:14.645573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0572f-6283-4c9a-ae0a-da9b2ba887aa None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.671708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:14.676325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd5677-c7b8-4f97-93bb-b12173a9db57 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.680788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.681477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.681879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.682162 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.682953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:14.683283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.684799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.697379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 381/758] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:14.702713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.714688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.715981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.716908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.717297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.727934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.729580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:14.748467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:14.753176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:14.754421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:14.754421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:14.754421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:14.754421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:14.754421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 378/759] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:14.754421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.810457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b6cde-a5ec-4afd-ad12-2f9e1a93c014 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.854733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.855613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.855996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.856371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.857286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:14.858142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.859608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.874567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556c534-0a36-40a0-ad3b-6a75e5611d5a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:14.877572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 382/760] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:14.878466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.901296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:14.913866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:14.928644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:14.928644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:14.928644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:14.928644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=9d71050d3f36438d8b2d2cd4c537d3c2) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:14.928644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:14.928644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:14.929574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 379/761] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/role_assignments?user.id=9d71050d3f36438d8b2d2cd4c537d3c2&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:14.929574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:14.965088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:15.018499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4315c5e-26d1-459c-900c-3a34e1aba35a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:15.090985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:15.092226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.092512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.092791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.093532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:15.093844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:15.096398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:15.129324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 383/762] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:15.135334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:15.144031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e29af5-61dd-44ef-be9c-9994912713a0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:15.193633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50753991-9064-4ea8-9e14-f9d616ccbca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:15.194611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50753991-9064-4ea8-9e14-f9d616ccbca9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.195078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50753991-9064-4ea8-9e14-f9d616ccbca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.195552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50753991-9064-4ea8-9e14-f9d616ccbca9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.196593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50753991-9064-4ea8-9e14-f9d616ccbca9 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=7556ce4bf09d4113aec2037dd438cecb) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:15.197903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50753991-9064-4ea8-9e14-f9d616ccbca9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:15.200127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50753991-9064-4ea8-9e14-f9d616ccbca9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:15.216372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 380/763] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:14 2022] GET /identity/v3/role_assignments?user.id=7556ce4bf09d4113aec2037dd438cecb&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=007af6bd1b734517b934b4cac45faf2c => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:15.263688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:15.274115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d01087-3f69-4c5a-9993-be684953d3c5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:15.328450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d02be3-8355-4aae-91b4-80b731990b5c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.328782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d02be3-8355-4aae-91b4-80b731990b5c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.329008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d02be3-8355-4aae-91b4-80b731990b5c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.330640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 381/764] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:15.330807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:15.331636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.331918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.332194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.335965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f1a48-c150-4449-8d63-13964af25eec None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.336251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f1a48-c150-4449-8d63-13964af25eec None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.336528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f1a48-c150-4449-8d63-13964af25eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.340793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:15.342651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:15.371392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a85f1a48-c150-4449-8d63-13964af25eec None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:15.372636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a85f1a48-c150-4449-8d63-13964af25eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:15.376830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:15.377741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:15.378647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:15.378647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:15.378647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:15.378647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:15.378647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:15.378647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:15.378647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:15.378647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:15.382696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 384/765] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:15.385057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:15.482606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 382/766] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:15.504718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f1a48-c150-4449-8d63-13964af25eec None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:15.526291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:15.526980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce8316-3c16-4f09-a150-244f2c47c70e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:15.566194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:15.566991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.567315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.567601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.568422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:15.568956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:15.570381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:15.583468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 385/767] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:15.583921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:15.591571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:15.685554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f1a48-c150-4449-8d63-13964af25eec None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:15.725079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:15.726334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.727517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.729264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.730093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:15.731820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:15.761524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:15.762714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:15.763746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:15.771804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 383/768] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 11 11:06:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:15.772859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:15.795705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a8f3e-13fb-4d30-9fd8-6ab35179d1ac None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:15.848801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fad1c-6ff5-4d44-bd79-4a24ecf0365b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:15.849686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fad1c-6ff5-4d44-bd79-4a24ecf0365b None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.849968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fad1c-6ff5-4d44-bd79-4a24ecf0365b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.850241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fad1c-6ff5-4d44-bd79-4a24ecf0365b None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/258ce5f4d6ff400199a80157cc34f5e5/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.855479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fad1c-6ff5-4d44-bd79-4a24ecf0365b None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=258ce5f4d6ff400199a80157cc34f5e5, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:15.856914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fad1c-6ff5-4d44-bd79-4a24ecf0365b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:15.896937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:15.928125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78da6b-4c01-414c-971d-5d73365f89c8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:15.937809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 386/769] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:06:15 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/258ce5f4d6ff400199a80157cc34f5e5/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:06:15.965752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:15.966863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:15.967385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:15.967909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:15.968836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:15.969341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:15.971088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:15.977118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 384/770] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 11 11:06:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:15.983724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fad1c-6ff5-4d44-bd79-4a24ecf0365b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:16.093860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:16.160619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fad1c-6ff5-4d44-bd79-4a24ecf0365b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:16.214010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7bb08-d590-4d7c-8081-d98862bd13c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:16.214688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bb08-d590-4d7c-8081-d98862bd13c2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:16.214971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bb08-d590-4d7c-8081-d98862bd13c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:16.215263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7bb08-d590-4d7c-8081-d98862bd13c2 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:16.215991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7bb08-d590-4d7c-8081-d98862bd13c2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:16.216300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7bb08-d590-4d7c-8081-d98862bd13c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:16.217666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7bb08-d590-4d7c-8081-d98862bd13c2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:16.228021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 387/771] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 11 11:06:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:16.233678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:16.351591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d4a89-5412-4c90-b580-9de6c8cc2752 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:16.351790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d4a89-5412-4c90-b580-9de6c8cc2752 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:16.352074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d4a89-5412-4c90-b580-9de6c8cc2752 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:16.354872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 388/772] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:16.362291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e09ea-f3c1-40ee-bfb2-a067adaee78b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:16.362586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e09ea-f3c1-40ee-bfb2-a067adaee78b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:16.363142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e09ea-f3c1-40ee-bfb2-a067adaee78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:16.367773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633ab87-44ea-4965-977b-ac9762c59dfa None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:16.384511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b86e09ea-f3c1-40ee-bfb2-a067adaee78b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:16.386437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b86e09ea-f3c1-40ee-bfb2-a067adaee78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:16.433176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:16.434437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:16.434922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:16.435440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:16.436484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] RBAC: Adding query filter params (interface=public, service_id=c0612a163cb946b6b1e52afbc4cdbe23, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:16.436803 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:16.438542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:16.447573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:16.454995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 385/773] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 11 11:06:16 2022] GET /identity/v3/endpoints?service_id=c0612a163cb946b6b1e52afbc4cdbe23&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:16.458837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:16.483661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 389/774] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:16.504207 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e09ea-f3c1-40ee-bfb2-a067adaee78b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:16.571636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:16.622542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e09ea-f3c1-40ee-bfb2-a067adaee78b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:16.661945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc2915-11f4-4ecd-8cc4-2bf57baf4a98 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:16.678506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:16.681095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:16.681430 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:16.681711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] PATH_INFO: `/v3/services/network` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:16.682376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:16.687586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:16.713562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:16.714662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:16.716075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:16.719602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 390/775] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:16 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:16.722988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:16.736608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becdc3a-05d7-4e26-bf6b-b92d726d655d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:16.737286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becdc3a-05d7-4e26-bf6b-b92d726d655d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:16.737569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becdc3a-05d7-4e26-bf6b-b92d726d655d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:16.737845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becdc3a-05d7-4e26-bf6b-b92d726d655d None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:16.738537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becdc3a-05d7-4e26-bf6b-b92d726d655d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:16.740025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becdc3a-05d7-4e26-bf6b-b92d726d655d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:16.750340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 386/776] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:06:16 2022] GET /identity/v3/services => generated 772 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:16.871426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90514f-db6e-48a3-816a-c7260fcfffdb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:16.889914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:16.908522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:16.909188 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:16.909474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:16.909750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:16.910465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:16.910827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:16.912227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:16.917607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 391/777] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:16.923563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becdc3a-05d7-4e26-bf6b-b92d726d655d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:17.019666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:17.056274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becdc3a-05d7-4e26-bf6b-b92d726d655d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:17.098329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:17.099006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.099320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.099609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.100327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] RBAC: Adding query filter params (type=network) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:17.100641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:17.102014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:17.107517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 387/778] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:17.113247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:17.279405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc784d07-4192-419d-b7f4-fa6255e18ad6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:17.314002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff163a63-5d0e-4429-8eaa-be4345ea2e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:17.314718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff163a63-5d0e-4429-8eaa-be4345ea2e65 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.315008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff163a63-5d0e-4429-8eaa-be4345ea2e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.315300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff163a63-5d0e-4429-8eaa-be4345ea2e65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.316102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff163a63-5d0e-4429-8eaa-be4345ea2e65 None admin] RBAC: Adding query filter params (interface=public, service_id=14eb3b7df14d4d10b563721d274a5dc5, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:17.316407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff163a63-5d0e-4429-8eaa-be4345ea2e65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:17.317761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff163a63-5d0e-4429-8eaa-be4345ea2e65 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:17.326312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 392/779] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 11 11:06:17 2022] GET /identity/v3/endpoints?service_id=14eb3b7df14d4d10b563721d274a5dc5&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:17.331844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:17.473581 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4b2bd-e2df-4378-9f72-d861e86ba136 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:17.512028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:17.520836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1b921-0f48-4d9c-8578-5231af6cc7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:17.521519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1b921-0f48-4d9c-8578-5231af6cc7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.521811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1b921-0f48-4d9c-8578-5231af6cc7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.522089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1b921-0f48-4d9c-8578-5231af6cc7f1 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.522776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1b921-0f48-4d9c-8578-5231af6cc7f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:17.524169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1b921-0f48-4d9c-8578-5231af6cc7f1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:17.532170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 388/780] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:06:17 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:17.595821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fde2af-97d5-404d-8d9e-2629c8ebfb6a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.596149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fde2af-97d5-404d-8d9e-2629c8ebfb6a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.596373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fde2af-97d5-404d-8d9e-2629c8ebfb6a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.598119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 393/781] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:17.603940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd64b-2251-4c35-ba63-8d17153fd5d3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.604237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd64b-2251-4c35-ba63-8d17153fd5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.604520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd64b-2251-4c35-ba63-8d17153fd5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.629639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-968cd64b-2251-4c35-ba63-8d17153fd5d3 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:17.630877 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-968cd64b-2251-4c35-ba63-8d17153fd5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:17.631594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66d813-4b62-4dfe-9418-9ce4a77f781a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.631885 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66d813-4b62-4dfe-9418-9ce4a77f781a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.632164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66d813-4b62-4dfe-9418-9ce4a77f781a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.633832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 394/782] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:17.639990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b56f0-d9db-41cb-969f-af8696c59b70 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.640313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b56f0-d9db-41cb-969f-af8696c59b70 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.640540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b56f0-d9db-41cb-969f-af8696c59b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.658027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c01b56f0-d9db-41cb-969f-af8696c59b70 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:17.659239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c01b56f0-d9db-41cb-969f-af8696c59b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:17.713742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:17.743612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 395/783] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:17.752376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 389/784] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:17.762944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b56f0-d9db-41cb-969f-af8696c59b70 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:17.775794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd64b-2251-4c35-ba63-8d17153fd5d3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:17.873846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b56f0-d9db-41cb-969f-af8696c59b70 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:17.889010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd64b-2251-4c35-ba63-8d17153fd5d3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:17.909652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4efddc-6bb5-49d6-893b-9bab1620c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:17.910439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4efddc-6bb5-49d6-893b-9bab1620c558 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.910716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4efddc-6bb5-49d6-893b-9bab1620c558 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.910986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4efddc-6bb5-49d6-893b-9bab1620c558 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.921873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4efddc-6bb5-49d6-893b-9bab1620c558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:17.923597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1e4efddc-6bb5-49d6-893b-9bab1620c558 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:17.929798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:17.939808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:17.941837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:17.943362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da737754-193a-4750-8463-68e6616e1635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:17.943545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 396/785] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:17.943689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da737754-193a-4750-8463-68e6616e1635 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:17.943689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da737754-193a-4750-8463-68e6616e1635 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:17.943689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da737754-193a-4750-8463-68e6616e1635 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:17.944096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4efddc-6bb5-49d6-893b-9bab1620c558 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:17.952421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da737754-193a-4750-8463-68e6616e1635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:17.974531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-da737754-193a-4750-8463-68e6616e1635 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:17.976261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:17.978833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:17.980796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 390/786] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:17.988247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da737754-193a-4750-8463-68e6616e1635 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.040769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4efddc-6bb5-49d6-893b-9bab1620c558 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:18.077617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:18.078294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.078574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.078846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.079634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:18.079945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:18.081331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:18.090738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 397/787] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:18.096132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.185425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da737754-193a-4750-8463-68e6616e1635 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:18.199789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7f437-7c23-4107-b951-bde0e693a856 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:18.203588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:18.237588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:18.238265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.238551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.238829 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.240755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:18.241053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:18.242423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:18.249422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:18.250220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.250506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.250783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.265134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 391/788] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:06:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:18.273368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:18.274835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:18.276473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:18.293429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:18.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:18.296592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:18.296592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:18.296592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:18.296592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:18.296592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 398/789] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:18.300442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.351691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:18.442631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2523-4623-4a5a-9630-a675c9d04055 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:18.456339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8e5d7-897e-4801-9a0a-6f90cdccc576 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:18.478795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:18.479613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.479896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.480166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.491360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:18.492764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:18.495832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4657-d73f-4fc7-a7a2-50f6251d1150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:18.496598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4657-d73f-4fc7-a7a2-50f6251d1150 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.496999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4657-d73f-4fc7-a7a2-50f6251d1150 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.497375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4657-d73f-4fc7-a7a2-50f6251d1150 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.498205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4657-d73f-4fc7-a7a2-50f6251d1150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:18.498616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4657-d73f-4fc7-a7a2-50f6251d1150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:18.500108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4657-d73f-4fc7-a7a2-50f6251d1150 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:18.511788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:18.512954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:18.514053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:18.514053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:18.514053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:18.514053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:18.514811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 399/790] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:18.516036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 392/791] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:18.520339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dbe7e-ac26-4e7e-adcb-7de71f829019 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.520339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dbe7e-ac26-4e7e-adcb-7de71f829019 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.520339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dbe7e-ac26-4e7e-adcb-7de71f829019 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.521551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.522417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 400/792] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:18.523165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68dbe7e-ac26-4e7e-adcb-7de71f829019 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.536877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:18.627533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d968b-57ce-4be4-9307-8c35303545bf None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:18.635477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:18.636105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68dbe7e-ac26-4e7e-adcb-7de71f829019 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:18.663995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:18.665336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.665966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.666457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.676481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3be29-6f66-44d5-b2c0-04051462f412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:18.677098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3be29-6f66-44d5-b2c0-04051462f412 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.677386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3be29-6f66-44d5-b2c0-04051462f412 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.677659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3be29-6f66-44d5-b2c0-04051462f412 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.678378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3be29-6f66-44d5-b2c0-04051462f412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:18.678840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3be29-6f66-44d5-b2c0-04051462f412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:18.679677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:18.680353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3be29-6f66-44d5-b2c0-04051462f412 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:18.681095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:18.693990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 401/793] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:18.695289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aceaaa-52eb-4a93-ad3b-1b480de64ab5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.695587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aceaaa-52eb-4a93-ad3b-1b480de64ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.695860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aceaaa-52eb-4a93-ad3b-1b480de64ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:18.699612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:18.700523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:18.701464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:18.702456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 393/794] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:18.707288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.719136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-19aceaaa-52eb-4a93-ad3b-1b480de64ab5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:18.720368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-19aceaaa-52eb-4a93-ad3b-1b480de64ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:18.797325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:18.807465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 402/795] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:18.808649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aceaaa-52eb-4a93-ad3b-1b480de64ab5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.819290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469d755-8314-4b39-bef5-6178433bbcbb None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:18.849808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:18.850621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:18.850903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:18.851178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:18.859665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:18.861010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:18.876684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:18.877709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:18.878859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 394/796] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:06:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:18.879875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:18.902544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:18.955342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aceaaa-52eb-4a93-ad3b-1b480de64ab5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.019759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.020122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.023273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.023273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.023273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:19.023273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.023273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:19.047138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3fe951-e08a-442d-9579-14c42e1d0e1e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.063433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 403/797] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 11 11:06:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:19.063433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.095324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b548c4-06d4-47ca-a064-fdc59e6e9166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.096139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b548c4-06d4-47ca-a064-fdc59e6e9166 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.096423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b548c4-06d4-47ca-a064-fdc59e6e9166 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.096705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b548c4-06d4-47ca-a064-fdc59e6e9166 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.115284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b548c4-06d4-47ca-a064-fdc59e6e9166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-40b548c4-06d4-47ca-a064-fdc59e6e9166 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:19.120487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:19.121688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:19.122762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:19.127475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 395/798] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:19.128755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b548c4-06d4-47ca-a064-fdc59e6e9166 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.190764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:19.254037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:19.258210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d23c1d-f699-4971-a6ce-c8d0e0f865f3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.279959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b548c4-06d4-47ca-a064-fdc59e6e9166 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.320348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.321310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.321689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.322134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.327027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.327735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.328022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.328303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.329150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:19.329465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.330901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:19.333893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.335391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:19.360278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 404/799] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:19.360621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:19.362570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:19.364771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:19.364771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:19.364771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:19.364771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:19.365279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.375281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 396/800] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:19.375281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.490617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6adfd5-6f83-4825-8024-deaac39ff1e9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.517474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df360b-fa28-4866-8f8b-be42a5437114 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.527625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:19.533373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:19.536948 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.538172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.539613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.539613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.549956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.551436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:19.551645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dd565-b5ab-4ae7-9bc4-78c51c051785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.552303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd565-b5ab-4ae7-9bc4-78c51c051785 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.552584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd565-b5ab-4ae7-9bc4-78c51c051785 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.552854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dd565-b5ab-4ae7-9bc4-78c51c051785 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.553762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dd565-b5ab-4ae7-9bc4-78c51c051785 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:19.554072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dd565-b5ab-4ae7-9bc4-78c51c051785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.555508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dd565-b5ab-4ae7-9bc4-78c51c051785 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:19.568384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 405/801] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:19.568384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f13c01-005b-4c46-8e2a-291eb07e9c7f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.568384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f13c01-005b-4c46-8e2a-291eb07e9c7f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.568384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f13c01-005b-4c46-8e2a-291eb07e9c7f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.568384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 406/802] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:19.568384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f13c01-005b-4c46-8e2a-291eb07e9c7f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:19.573477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:19.574350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:19.575354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:19.575354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:19.575354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:19.575354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:19.582118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 397/803] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:19.582118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.673682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f13c01-005b-4c46-8e2a-291eb07e9c7f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.681480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a68ec-e5ed-43b6-bbc9-7faa9b1573fd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.711955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67a2ca-21c5-45e6-8b70-366284850115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.712811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a2ca-21c5-45e6-8b70-366284850115 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.713095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a2ca-21c5-45e6-8b70-366284850115 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.713369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a2ca-21c5-45e6-8b70-366284850115 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.714153 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a2ca-21c5-45e6-8b70-366284850115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:19.714489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a2ca-21c5-45e6-8b70-366284850115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.715996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.716685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a2ca-21c5-45e6-8b70-366284850115 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:19.716782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.717063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.717341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.730560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.732047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:19.735390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 407/804] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:19.736915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0f21-1f8c-4993-b861-3cff925b1150 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.737215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0f21-1f8c-4993-b861-3cff925b1150 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.737509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b0f21-1f8c-4993-b861-3cff925b1150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:19.750108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:19.751207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:19.756994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:19.756994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:19.756994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:19.756994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:19.756994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 398/805] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:19.756994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.760565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-292b0f21-1f8c-4993-b861-3cff925b1150 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:19.762175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-292b0f21-1f8c-4993-b861-3cff925b1150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:19.785624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:19.797173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:19.841304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 408/806] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:19.842001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b0f21-1f8c-4993-b861-3cff925b1150 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.881224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b154f3-49bd-4123-9037-6d42ff218f8e None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.918267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.918930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.919209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.919509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.920217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:19.920726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.921862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:19.938367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 399/807] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:19.938367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:19.946351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b0f21-1f8c-4993-b861-3cff925b1150 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:19.985629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:19.986423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:19.986704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:19.986984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:19.994538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:19.996024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:20.010257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:20.011340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:20.012460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:20.012460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:20.012460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:20.012460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:20.012460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:20.012460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:20.012460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:20.012460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:20.015683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 409/808] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:20.017131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.045284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:20.047061 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:20.060079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9df757-912d-4c02-a912-7c7683988c34 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.093216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.094019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.094403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.094952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.095802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:20.096230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.097695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.117549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 400/809] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:20.117549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.140679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba252d-14c9-41df-8d7b-b6a892df4bd5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.181064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.181911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.182195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.182469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.183117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.184514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:20.196133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:20.197223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 11:06:20.198320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:20.202291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 410/810] 10.222.0.26 () {56 vars in 1129 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:20.203048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.228991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1644bb5-eca9-41ce-9d77-7340b1612f11 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.260918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.261706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.261986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.262260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.268637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.270048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:20.284082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:20.285164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 401/811] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:20.291774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.299636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:20.301311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:20.307806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c299651-12de-4d31-a4f4-ba0350d3b597 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.339842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dddf6a-daea-4b9f-97d6-9f875f3a387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.340493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dddf6a-daea-4b9f-97d6-9f875f3a387e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.340771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dddf6a-daea-4b9f-97d6-9f875f3a387e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.341043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dddf6a-daea-4b9f-97d6-9f875f3a387e None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.341819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dddf6a-daea-4b9f-97d6-9f875f3a387e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:20.342126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dddf6a-daea-4b9f-97d6-9f875f3a387e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.343685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dddf6a-daea-4b9f-97d6-9f875f3a387e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.355730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 411/812] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:20.357220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd059b92-5c1c-41eb-ad28-fa256e6dbf74 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.357505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd059b92-5c1c-41eb-ad28-fa256e6dbf74 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.357780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd059b92-5c1c-41eb-ad28-fa256e6dbf74 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.358992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 412/813] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:20.359702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd059b92-5c1c-41eb-ad28-fa256e6dbf74 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.386805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e89ff-6a29-493a-a58a-172b73fe26bf None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.419129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.419940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.420225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.420501 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.427975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.429395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:20.447584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:20.448800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 402/814] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:20.450083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.465370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd059b92-5c1c-41eb-ad28-fa256e6dbf74 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.499195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.499872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.500152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.500423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.501121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:20.501423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.502769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.507900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 413/815] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:20.508662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.524238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:20.525553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:20.554229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a42428-6d8b-49ab-9e36-743ff2903722 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.587731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9fc22-3763-4240-b5e9-03056df1cad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.588391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9fc22-3763-4240-b5e9-03056df1cad8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.588676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9fc22-3763-4240-b5e9-03056df1cad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.588951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9fc22-3763-4240-b5e9-03056df1cad8 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.589730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9fc22-3763-4240-b5e9-03056df1cad8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:20.590038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9fc22-3763-4240-b5e9-03056df1cad8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.591493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9fc22-3763-4240-b5e9-03056df1cad8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.603537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 403/816] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:20.604714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4d4d5-f007-4069-b6a8-047446588c7e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.604995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4d4d5-f007-4069-b6a8-047446588c7e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.605270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4d4d5-f007-4069-b6a8-047446588c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.605491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e35c1f-24d0-4c5f-a87a-3a67ccccac10 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.623056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-75e4d4d5-f007-4069-b6a8-047446588c7e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:20.624259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-75e4d4d5-f007-4069-b6a8-047446588c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:20.639766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91843340-a9ac-4005-ae78-9019e4ef7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.640650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91843340-a9ac-4005-ae78-9019e4ef7ad6 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.640932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91843340-a9ac-4005-ae78-9019e4ef7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.641210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91843340-a9ac-4005-ae78-9019e4ef7ad6 None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/7556ce4bf09d4113aec2037dd438cecb/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.643169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91843340-a9ac-4005-ae78-9019e4ef7ad6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=7556ce4bf09d4113aec2037dd438cecb, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.644605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91843340-a9ac-4005-ae78-9019e4ef7ad6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.687505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 404/817] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:20.688311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4d4d5-f007-4069-b6a8-047446588c7e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.735584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 414/818] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:06:20 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/7556ce4bf09d4113aec2037dd438cecb/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:06:20.736743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91843340-a9ac-4005-ae78-9019e4ef7ad6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.757414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:20.808554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4d4d5-f007-4069-b6a8-047446588c7e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.847403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:20.848015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.848783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.849140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.849487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.850453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:20.850839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.852431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.876931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 405/819] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:20.876931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.880108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91843340-a9ac-4005-ae78-9019e4ef7ad6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:20.920017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:20.920684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:20.920971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:20.921230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:20.921938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:20.922245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:20.923667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:20.929041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 415/820] 10.222.0.26 () {56 vars in 1124 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:20.929758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:20.992338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9f1ff-95bc-445f-b4fd-51468ab2a459 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:21.013716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:21.029529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af89e72-5be0-4f54-b558-a132695c747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:21.030885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af89e72-5be0-4f54-b558-a132695c747f None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:21.031376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af89e72-5be0-4f54-b558-a132695c747f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:21.031832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af89e72-5be0-4f54-b558-a132695c747f None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/9d71050d3f36438d8b2d2cd4c537d3c2/roles/d82425658c8545e6b546d601ef7b9b58` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:21.041710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288a25c-684e-401e-84c0-3849aaf26774 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:21.048816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af89e72-5be0-4f54-b558-a132695c747f None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=9d71050d3f36438d8b2d2cd4c537d3c2, role_id=d82425658c8545e6b546d601ef7b9b58)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:21.050979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af89e72-5be0-4f54-b558-a132695c747f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:21.081547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:21.082431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:21.082826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:21.083171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] PATH_INFO: `/v3/services/network` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:21.083921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:21.085354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:21.098303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:21.107312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:21.108250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 416/821] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:20 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:21.109043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:21.109429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:21.137585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 406/822] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:06:20 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/9d71050d3f36438d8b2d2cd4c537d3c2/roles/d82425658c8545e6b546d601ef7b9b58 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:06:21.138674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af89e72-5be0-4f54-b558-a132695c747f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:21.239977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde7d6e-bb28-4b1a-9e2d-298378f621cd None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:21.279107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3856-ca91-4a35-b65e-e26a53d79553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:21.280226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3856-ca91-4a35-b65e-e26a53d79553 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:21.280710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3856-ca91-4a35-b65e-e26a53d79553 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:21.281183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd3856-ca91-4a35-b65e-e26a53d79553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:21.293639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af89e72-5be0-4f54-b558-a132695c747f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:21.335223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4082-ad3b-470a-930d-fab383342f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:21.335849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4082-ad3b-470a-930d-fab383342f18 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:21.336136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4082-ad3b-470a-930d-fab383342f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:21.336413 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad4082-ad3b-470a-930d-fab383342f18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:21.337156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad4082-ad3b-470a-930d-fab383342f18 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:21.338554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad4082-ad3b-470a-930d-fab383342f18 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:21.360351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3856-ca91-4a35-b65e-e26a53d79553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:21.361805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd3856-ca91-4a35-b65e-e26a53d79553 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:21.365633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:21.441286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 407/823] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:06:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:21.442708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4082-ad3b-470a-930d-fab383342f18 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:21.481209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:21.498717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 417/824] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:06:21.504760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3856-ca91-4a35-b65e-e26a53d79553 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:21.577500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad4082-ad3b-470a-930d-fab383342f18 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:21.632941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:21.633542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:21.633827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:21.634163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:21.634955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] RBAC: Adding query filter params (name=network) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:21.635339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:21.637445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:21.643017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 408/825] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:21.648588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:21.718698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd3856-ca91-4a35-b65e-e26a53d79553 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:21.722664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:21.772805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:21.783307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:21.783599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:21.785950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:21.786252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:21.796629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eccb7c-a2f0-43cc-92ca-a1c5c20b8304 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:21.800247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:21.801824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:21.821100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:21.822581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:21.823995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:21.823995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:21.823995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:21.823995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:21.827431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 418/826] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:21.833042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:21.838487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:21.839475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:21.839964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:21.840440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:21.841386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] RBAC: Adding query filter params (type=network) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:21.841898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:21.843563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:21.849688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 409/827] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:21.857534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:22.009909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025910a2-2629-4174-b75e-4cc97eeb8c34 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:22.120940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:22.121631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:22.121918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:22.122200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:22.122946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:22.123276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:22.124678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:22.150610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4da6c-03cc-4bf3-89ee-82a130fd7796 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:22.153728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 419/828] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:22.158908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:22.188042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529f1c-064e-4341-90b9-6a2f545a3d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:22.188711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529f1c-064e-4341-90b9-6a2f545a3d6a None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:22.189002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529f1c-064e-4341-90b9-6a2f545a3d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:22.189282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529f1c-064e-4341-90b9-6a2f545a3d6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:22.190029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce529f1c-064e-4341-90b9-6a2f545a3d6a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:22.191536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce529f1c-064e-4341-90b9-6a2f545a3d6a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:22.199576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:22.285752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40555242-2157-4357-b2cd-aeda297ae01e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:22.313685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 410/829] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:06:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:06:22.329512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:22.330349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:22.330636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:22.330966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:22.339476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:22.341029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:22.486144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:22.487588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:22.488953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 420/830] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:22.493068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529f1c-064e-4341-90b9-6a2f545a3d6a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:22.514444 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:22.645086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529f1c-064e-4341-90b9-6a2f545a3d6a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:22.680211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173181b-d6a5-43a2-aa15-7648f2e47538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:22.680875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173181b-d6a5-43a2-aa15-7648f2e47538 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:22.681215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173181b-d6a5-43a2-aa15-7648f2e47538 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:22.681491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173181b-d6a5-43a2-aa15-7648f2e47538 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:22.682273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173181b-d6a5-43a2-aa15-7648f2e47538 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:22.682580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173181b-d6a5-43a2-aa15-7648f2e47538 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:22.684019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173181b-d6a5-43a2-aa15-7648f2e47538 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:22.696081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 411/831] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:22.702398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:22.783489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:22.816151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb01b-ff2f-437c-bbda-51cf998dc233 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:22.851765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca7fc3-8277-4e3f-b55e-53a8825679d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:22.852417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7fc3-8277-4e3f-b55e-53a8825679d7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:22.852700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7fc3-8277-4e3f-b55e-53a8825679d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:22.853493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca7fc3-8277-4e3f-b55e-53a8825679d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:22.854320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7fc3-8277-4e3f-b55e-53a8825679d7 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=258ce5f4d6ff400199a80157cc34f5e5) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:22.854629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7fc3-8277-4e3f-b55e-53a8825679d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:22.856139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca7fc3-8277-4e3f-b55e-53a8825679d7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:22.862411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 421/832] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:22 2022] GET /identity/v3/role_assignments?user.id=258ce5f4d6ff400199a80157cc34f5e5&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=007af6bd1b734517b934b4cac45faf2c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:22.870949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7889de-5531-4735-a3a6-4acde4eeafea None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:22.871240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7889de-5531-4735-a3a6-4acde4eeafea None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:22.871550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7889de-5531-4735-a3a6-4acde4eeafea None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:22.873145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 412/833] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:22.881054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877f8d4-5800-45dc-867e-f1749d05fbe0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:22.881345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877f8d4-5800-45dc-867e-f1749d05fbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:22.881614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877f8d4-5800-45dc-867e-f1749d05fbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:22.899494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6877f8d4-5800-45dc-867e-f1749d05fbe0 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:22.900665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6877f8d4-5800-45dc-867e-f1749d05fbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:22.987420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 422/834] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:23.012393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7889de-5531-4735-a3a6-4acde4eeafea None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:23.084101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:23.121161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7889de-5531-4735-a3a6-4acde4eeafea None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:23.156125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848474b6-a60f-41a1-a59b-21822a663f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:23.156850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848474b6-a60f-41a1-a59b-21822a663f0b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.157114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848474b6-a60f-41a1-a59b-21822a663f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.157385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848474b6-a60f-41a1-a59b-21822a663f0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.168308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848474b6-a60f-41a1-a59b-21822a663f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-848474b6-a60f-41a1-a59b-21822a663f0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:23.176602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:23.177552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:23.178407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 413/835] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:23.188416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877f8d4-5800-45dc-867e-f1749d05fbe0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:23.260318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:23.291002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877f8d4-5800-45dc-867e-f1749d05fbe0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:23.327016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:23.327871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.328158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.328435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.329189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:23.329497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:23.330848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:23.339851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 423/836] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:23.345370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848474b6-a60f-41a1-a59b-21822a663f0b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:23.448759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848474b6-a60f-41a1-a59b-21822a663f0b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:23.479658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:23.485755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c71e51-4668-4e2b-9144-a880cec93385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:23.486714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71e51-4668-4e2b-9144-a880cec93385 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.487001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71e51-4668-4e2b-9144-a880cec93385 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.487294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c71e51-4668-4e2b-9144-a880cec93385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.499493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c71e51-4668-4e2b-9144-a880cec93385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:23.500989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c71e51-4668-4e2b-9144-a880cec93385 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f8c71e51-4668-4e2b-9144-a880cec93385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:23.518967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:23.519957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:23.520860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:23.520860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:23.520860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:23.520860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:23.520860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 414/837] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:23.526941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:23.632062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d91dcf-a491-41e6-b6db-623b8beb2c02 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:23.646551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4860-a638-4feb-9da1-1d3e42917593 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.647150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4860-a638-4feb-9da1-1d3e42917593 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.647681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4860-a638-4feb-9da1-1d3e42917593 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.649523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 415/838] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:23.655526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaf7d5-7fe0-4756-b0ac-209e5d410424 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.656082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaf7d5-7fe0-4756-b0ac-209e5d410424 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.656557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaaf7d5-7fe0-4756-b0ac-209e5d410424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.660985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:23.669508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:23.670304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.670585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.670864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.671604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:23.671906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:23.673256 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:23.676134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bcaaf7d5-7fe0-4756-b0ac-209e5d410424 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:23.682099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bcaaf7d5-7fe0-4756-b0ac-209e5d410424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:23.686968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 424/839] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:23.692663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:23.771876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 416/840] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:23.797240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaf7d5-7fe0-4756-b0ac-209e5d410424 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:23.798718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dda9c5-dcb4-4890-8328-3b4b21741052 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:23.834238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:23.834568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.834826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.835096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.843450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:23.844941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:23.861266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:23.862268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:23.863214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 425/841] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:23.868362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:23.914064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaaf7d5-7fe0-4756-b0ac-209e5d410424 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:23.919880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:23.924218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:23.949410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482725a4-8433-497c-8482-02cec5d36442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:23.952622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482725a4-8433-497c-8482-02cec5d36442 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.952622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482725a4-8433-497c-8482-02cec5d36442 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.952622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482725a4-8433-497c-8482-02cec5d36442 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.960758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482725a4-8433-497c-8482-02cec5d36442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-482725a4-8433-497c-8482-02cec5d36442 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:23.968762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:23.969701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:06:23.979483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:23.980278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641451a-e955-4a63-aab5-9dde37f60a47 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:23.980359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 417/842] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:06:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:23.980359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585166e-8427-4820-945b-f01a74233f8f None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:23.980359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585166e-8427-4820-945b-f01a74233f8f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:23.980359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585166e-8427-4820-945b-f01a74233f8f None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:23.980359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 418/843] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:23.980359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585166e-8427-4820-945b-f01a74233f8f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.011942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b22fc0-813a-4bee-8f11-e294d96f9b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.012716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b22fc0-813a-4bee-8f11-e294d96f9b8f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.013002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b22fc0-813a-4bee-8f11-e294d96f9b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.013278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b22fc0-813a-4bee-8f11-e294d96f9b8f None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.014138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b22fc0-813a-4bee-8f11-e294d96f9b8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:24.014406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b22fc0-813a-4bee-8f11-e294d96f9b8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.015868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b22fc0-813a-4bee-8f11-e294d96f9b8f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.038176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 426/844] 10.222.0.26 () {56 vars in 1135 bytes} [Mon Jul 11 11:06:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:24.038176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac1aac-2f9e-465f-b243-56042904b666 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.038176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac1aac-2f9e-465f-b243-56042904b666 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.038176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac1aac-2f9e-465f-b243-56042904b666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.054575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b7ac1aac-2f9e-465f-b243-56042904b666 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:24.055776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b7ac1aac-2f9e-465f-b243-56042904b666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:24.096156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585166e-8427-4820-945b-f01a74233f8f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.139280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 427/845] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:24.139280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac1aac-2f9e-465f-b243-56042904b666 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.150952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.152187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.152474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.153018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.153775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:24.154399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.156628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.166655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 419/846] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:06:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:24.166797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:24.171748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.202928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:24.235218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac1aac-2f9e-465f-b243-56042904b666 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.274062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af44165-26fb-4f56-af88-ba7da9185493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.274849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af44165-26fb-4f56-af88-ba7da9185493 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.275285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af44165-26fb-4f56-af88-ba7da9185493 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.275414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af44165-26fb-4f56-af88-ba7da9185493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.278137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a2d7e-dbab-4cf3-9304-e359519fa1e4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.315182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.315935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.316217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.316493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.317140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.318492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:24.330579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:24.331609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:24.332452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 420/847] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:24.333278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.349535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af44165-26fb-4f56-af88-ba7da9185493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.350966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af44165-26fb-4f56-af88-ba7da9185493 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.385700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:24.431200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:24.437756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f490610-ecb7-42ea-8281-b8520c5e1055 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.451621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 428/848] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:06:24.452275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af44165-26fb-4f56-af88-ba7da9185493 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.477063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.478348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.478867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.479375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.490976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.492941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:24.511776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:24.512749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:24.513867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:24.513867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:24.513867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:24.513867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:24.515483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 421/849] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:24.516380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.555719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af44165-26fb-4f56-af88-ba7da9185493 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.592884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.593754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.594181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.594613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.595494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:24.595960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.597446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.599653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:24.604201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 429/850] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:24.605059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.626265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e4f8f-b80f-4708-bf95-cce1e1b063f7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.661939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.663099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.663574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.663849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.675655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.677075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.685434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:24.696912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:24.701199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:24.702302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:24.702302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:24.702302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:24.702302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:24.702975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 422/851] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:24.703849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.713599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a4a36-3165-40e0-b840-e09c7db645d5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.750389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.755396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.755396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.755396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.755396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:24.755396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.755396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.766081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 430/852] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:24.767051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.813491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcfc86-ded5-43cf-b52f-a3bdd47056a4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.831976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:24.849368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.850320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.850602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.850874 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.851901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:24.852209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.854182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.858871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 423/853] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:24.872317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.875373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b316d6-5cf9-4d2c-a607-82e759c53e32 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:24.917053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:24.917740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:24.918025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:24.918305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:24.919216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:24.919464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:24.920933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:24.935765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 431/854] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:24.941563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:24.960047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:25.019981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51349d73-7b68-4e5f-8067-6042b9a0bb6a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.064551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914be18c-5dbb-43f4-8a94-aae20eb27941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.065373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914be18c-5dbb-43f4-8a94-aae20eb27941 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.065655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914be18c-5dbb-43f4-8a94-aae20eb27941 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.065923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914be18c-5dbb-43f4-8a94-aae20eb27941 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.074576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914be18c-5dbb-43f4-8a94-aae20eb27941 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.076048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914be18c-5dbb-43f4-8a94-aae20eb27941 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.076717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0019ab4-95e4-44b9-a268-3543bb7db387 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-914be18c-5dbb-43f4-8a94-aae20eb27941 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:25.090579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:25.091613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 424/855] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:25.093354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da2d5-2cfa-4304-9803-cee33f529400 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.094029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da2d5-2cfa-4304-9803-cee33f529400 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.094029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da2d5-2cfa-4304-9803-cee33f529400 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.095041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 425/856] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:25.099549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227da2d5-2cfa-4304-9803-cee33f529400 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.104615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:25.118809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42c8d7-dbd6-4f82-831b-0c2961d56177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.122042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c8d7-dbd6-4f82-831b-0c2961d56177 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.122042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c8d7-dbd6-4f82-831b-0c2961d56177 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.122042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c8d7-dbd6-4f82-831b-0c2961d56177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.127092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42c8d7-dbd6-4f82-831b-0c2961d56177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.128572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42c8d7-dbd6-4f82-831b-0c2961d56177 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4b42c8d7-dbd6-4f82-831b-0c2961d56177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:25.143029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:25.143998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:25.144919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:25.144919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:25.144919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:25.144919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:25.144919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:25.144919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:25.144919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:25.144919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:25.146728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 432/857] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:25.148068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b615-8173-4f8e-879a-47e82ff4f317 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.148351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b615-8173-4f8e-879a-47e82ff4f317 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.148627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261b615-8173-4f8e-879a-47e82ff4f317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.165092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9261b615-8173-4f8e-879a-47e82ff4f317 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:25.166775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9261b615-8173-4f8e-879a-47e82ff4f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:25.215374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227da2d5-2cfa-4304-9803-cee33f529400 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.232284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 433/858] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:25.233129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b615-8173-4f8e-879a-47e82ff4f317 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.247033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:25.259839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.260950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.261236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.261515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.262354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:25.262714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.264463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.288047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 426/859] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:25.288047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.335435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261b615-8173-4f8e-879a-47e82ff4f317 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.348540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:25.374175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.374866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.375153 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.375453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.376236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:25.376546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.377987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.396198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 434/860] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:25.396394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797415d-a9a4-41e2-b3dd-53e72df5ff37 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.396501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.430698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.432073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.432362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.432637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.433293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.434651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:25.446856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:25.448022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:25.448897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 427/861] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:25.449836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.497762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:25.505500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a2d0b-ce0f-47d0-b18e-72c1de0c5b0f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.541375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7f4f4-40e2-440c-9cae-6232230c69bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.542016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7f4f4-40e2-440c-9cae-6232230c69bc None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.542304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7f4f4-40e2-440c-9cae-6232230c69bc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.542575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7f4f4-40e2-440c-9cae-6232230c69bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.560237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ff886-c0d6-4750-a0ac-aa4f029f94a8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.584149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:25.597742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.598443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.598804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.598994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.599848 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=7556ce4bf09d4113aec2037dd438cecb) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:25.600157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.601625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.615540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 428/862] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/role_assignments?user.id=7556ce4bf09d4113aec2037dd438cecb&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=007af6bd1b734517b934b4cac45faf2c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:25.615540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.624663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7f4f4-40e2-440c-9cae-6232230c69bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.626238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7f4f4-40e2-440c-9cae-6232230c69bc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.748581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 435/863] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:06:25.756310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7f4f4-40e2-440c-9cae-6232230c69bc None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.756448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044fa1-a19b-4672-9df1-2bdd8adeb6f5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.767542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:25.809859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.810825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.811116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.811468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.812355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:25.812676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.814715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.823293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 429/864] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:25.827764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.843734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:25.886024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7f4f4-40e2-440c-9cae-6232230c69bc None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.926047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.927172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:25.927688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:25.928154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:25.942257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:25.944808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:25.955337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bcf17-179f-417e-9975-a04ba5f7a707 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:25.969423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:25.970631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:25.971880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:25.971880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:25.971880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:25.971880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:25.974590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 436/865] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:25.980096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:25.999068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:25.999935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:26.000224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:26.000507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:26.001227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:26.001542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:26.003437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:26.015319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 430/866] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:26.099912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4849ce-1562-47f5-9b98-a2d33bd3a85a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:26.126385 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:26.138614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:26.139299 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:26.139595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:26.139876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:26.140618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:26.140932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:26.142334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:26.156050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 437/867] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:26.161524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:26.201794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:26.267150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a248922-24cb-4014-914f-f8970a93e54b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:26.302534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:26.303749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:26.304245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:26.304722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:26.311870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:26.313541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:26.328692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:26.329699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:26.331463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:26.331463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:26.331463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:26.331463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:26.331463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:26.331463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:26.331463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:26.331463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:26.335741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 431/868] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:26.344635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:26.453962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb331d-2a82-498c-9d48-df9aee4137ce None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:26.488320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fd012-23e5-48b6-8490-b4b5e5d682ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:26.488980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd012-23e5-48b6-8490-b4b5e5d682ea None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:26.489262 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd012-23e5-48b6-8490-b4b5e5d682ea None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:26.489542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fd012-23e5-48b6-8490-b4b5e5d682ea None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:26.490327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fd012-23e5-48b6-8490-b4b5e5d682ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:26.490637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fd012-23e5-48b6-8490-b4b5e5d682ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:26.492141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fd012-23e5-48b6-8490-b4b5e5d682ea None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:26.504434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 438/869] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:26.510864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:26.616502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:26.623096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a13c78-f87b-4f44-8e46-f31811b72a28 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:26.657911 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d451dc9-408d-49a2-930d-feb64fce0cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:26.658570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d451dc9-408d-49a2-930d-feb64fce0cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:26.658855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d451dc9-408d-49a2-930d-feb64fce0cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:26.659135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d451dc9-408d-49a2-930d-feb64fce0cd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:26.660001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d451dc9-408d-49a2-930d-feb64fce0cd0 None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=9d71050d3f36438d8b2d2cd4c537d3c2) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:26.660647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d451dc9-408d-49a2-930d-feb64fce0cd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:26.662064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d451dc9-408d-49a2-930d-feb64fce0cd0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:26.668454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 432/870] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:26 2022] GET /identity/v3/role_assignments?user.id=9d71050d3f36438d8b2d2cd4c537d3c2&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:26.968880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd2fca-ab6b-42c6-85b4-eb4f0fa40a4a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:26.969738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd2fca-ab6b-42c6-85b4-eb4f0fa40a4a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:26.970183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd2fca-ab6b-42c6-85b4-eb4f0fa40a4a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:26.971961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 439/871] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:26.977425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0846ac-879c-4c6a-87e1-f2875b307a18 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:26.978596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0846ac-879c-4c6a-87e1-f2875b307a18 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:26.978885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0846ac-879c-4c6a-87e1-f2875b307a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:26.995588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ab0846ac-879c-4c6a-87e1-f2875b307a18 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:26.996757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ab0846ac-879c-4c6a-87e1-f2875b307a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:27.069875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 433/872] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:27.093199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd2fca-ab6b-42c6-85b4-eb4f0fa40a4a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:27.139461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:27.199592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd2fca-ab6b-42c6-85b4-eb4f0fa40a4a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:27.235845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c70aff-9781-4241-9c56-e5b606dec8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:27.236909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c70aff-9781-4241-9c56-e5b606dec8f7 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:27.237426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c70aff-9781-4241-9c56-e5b606dec8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:27.237913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c70aff-9781-4241-9c56-e5b606dec8f7 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:27.238864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c70aff-9781-4241-9c56-e5b606dec8f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:27.240556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c70aff-9781-4241-9c56-e5b606dec8f7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:27.317272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 440/873] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:06:27 2022] POST /identity/v3/services => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:27.767514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042918f-e8bb-4f36-8858-15953866996d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:27.768069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042918f-e8bb-4f36-8858-15953866996d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:27.768069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042918f-e8bb-4f36-8858-15953866996d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:27.769803 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 434/874] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:27.775690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a8d3-03ba-4de6-960d-b8af9014458e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:27.775986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a8d3-03ba-4de6-960d-b8af9014458e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:27.776266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499a8d3-03ba-4de6-960d-b8af9014458e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:27.794058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7499a8d3-03ba-4de6-960d-b8af9014458e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:27.795279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7499a8d3-03ba-4de6-960d-b8af9014458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:27.879432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 441/875] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:27.896439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042918f-e8bb-4f36-8858-15953866996d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:27.991609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:28.000556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042918f-e8bb-4f36-8858-15953866996d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:28.034732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3438e2-fb9d-491b-aef3-e019ec666ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:28.036001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3438e2-fb9d-491b-aef3-e019ec666ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:28.036486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3438e2-fb9d-491b-aef3-e019ec666ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:28.037180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3438e2-fb9d-491b-aef3-e019ec666ce0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:28.038022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3438e2-fb9d-491b-aef3-e019ec666ce0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:28.039596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3438e2-fb9d-491b-aef3-e019ec666ce0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3e3438e2-fb9d-491b-aef3-e019ec666ce0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:28.052952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:28.054002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:28.054847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:28.057346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 435/876] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:28.064995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499a8d3-03ba-4de6-960d-b8af9014458e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:28.098851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:28.173250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499a8d3-03ba-4de6-960d-b8af9014458e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:28.207513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:28.208413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:28.208850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:28.209269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:28.210119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:28.210571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:28.212623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:28.215614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772167b-b881-4aac-a884-6d528aa31a91 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:28.216331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772167b-b881-4aac-a884-6d528aa31a91 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:28.216815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772167b-b881-4aac-a884-6d528aa31a91 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:28.218840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 436/877] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:28.219129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 442/878] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:28.223919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71f289-884e-4856-9c8a-b2bd541c51ef None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:28.224208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71f289-884e-4856-9c8a-b2bd541c51ef None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:28.224489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71f289-884e-4856-9c8a-b2bd541c51ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:28.225136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:28.241442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4e71f289-884e-4856-9c8a-b2bd541c51ef None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:28.242733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4e71f289-884e-4856-9c8a-b2bd541c51ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:28.323793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 437/879] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:28.331540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:28.340895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71f289-884e-4856-9c8a-b2bd541c51ef None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:28.342984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da3b31-1165-4444-878a-07308bc568b5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:28.388590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bea547-70ef-45c0-9d53-28a3d9b19632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:28.389408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bea547-70ef-45c0-9d53-28a3d9b19632 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:28.389805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bea547-70ef-45c0-9d53-28a3d9b19632 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:28.390085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bea547-70ef-45c0-9d53-28a3d9b19632 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:28.390796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bea547-70ef-45c0-9d53-28a3d9b19632 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:28.391102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bea547-70ef-45c0-9d53-28a3d9b19632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:28.393212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bea547-70ef-45c0-9d53-28a3d9b19632 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:28.400255 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 443/880] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:28.442776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:28.459640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71f289-884e-4856-9c8a-b2bd541c51ef None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:28.500365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aee473-1fbd-46de-ae27-176ed6e1e092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:28.501052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee473-1fbd-46de-ae27-176ed6e1e092 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:28.501336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee473-1fbd-46de-ae27-176ed6e1e092 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:28.501613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee473-1fbd-46de-ae27-176ed6e1e092 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:28.502354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aee473-1fbd-46de-ae27-176ed6e1e092 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:28.503769 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aee473-1fbd-46de-ae27-176ed6e1e092 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:28.628608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 438/881] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:06:28 2022] POST /identity/v3/services => generated 269 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:06:28.850283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591bfd-2e23-48d3-a2e2-ac48a88ddd8f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:28.850699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591bfd-2e23-48d3-a2e2-ac48a88ddd8f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:28.850888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591bfd-2e23-48d3-a2e2-ac48a88ddd8f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:28.852371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 444/882] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:28.858308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156b927-ee96-4f37-a98b-36959e618fe2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:28.859219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156b927-ee96-4f37-a98b-36959e618fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:28.859725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156b927-ee96-4f37-a98b-36959e618fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:28.876993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1156b927-ee96-4f37-a98b-36959e618fe2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:28.878194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1156b927-ee96-4f37-a98b-36959e618fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:28.951452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 439/883] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:28.974697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb591bfd-2e23-48d3-a2e2-ac48a88ddd8f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.082413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb591bfd-2e23-48d3-a2e2-ac48a88ddd8f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:29.117770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348b3ac-f575-4cc4-b7ba-2560ce060d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:29.119109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348b3ac-f575-4cc4-b7ba-2560ce060d4b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.119417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348b3ac-f575-4cc4-b7ba-2560ce060d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.119688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348b3ac-f575-4cc4-b7ba-2560ce060d4b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.130861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348b3ac-f575-4cc4-b7ba-2560ce060d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c348b3ac-f575-4cc4-b7ba-2560ce060d4b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:29.138912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:29.139894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:29.140768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 445/884] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 11 11:06:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:29.146474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156b927-ee96-4f37-a98b-36959e618fe2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.218137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:29.248064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156b927-ee96-4f37-a98b-36959e618fe2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:29.281590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173ad7f-267a-4300-bab4-149f8b4c9128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:29.282330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173ad7f-267a-4300-bab4-149f8b4c9128 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.282613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173ad7f-267a-4300-bab4-149f8b4c9128 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.282886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173ad7f-267a-4300-bab4-149f8b4c9128 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.283658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173ad7f-267a-4300-bab4-149f8b4c9128 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:29.283964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173ad7f-267a-4300-bab4-149f8b4c9128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:29.285319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173ad7f-267a-4300-bab4-149f8b4c9128 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:29.294194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 440/885] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 11 11:06:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:29.299760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348b3ac-f575-4cc4-b7ba-2560ce060d4b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.318957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:29.401407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348b3ac-f575-4cc4-b7ba-2560ce060d4b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:29.435819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f4974-266e-484d-bfdc-54ff3ad11e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:29.436606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f4974-266e-484d-bfdc-54ff3ad11e17 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.436881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f4974-266e-484d-bfdc-54ff3ad11e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.437150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f4974-266e-484d-bfdc-54ff3ad11e17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.448344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f4974-266e-484d-bfdc-54ff3ad11e17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:29.449775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f4974-266e-484d-bfdc-54ff3ad11e17 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:29.463549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59235-558b-4eca-9f18-32f100a16c3c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.463843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59235-558b-4eca-9f18-32f100a16c3c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.464129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59235-558b-4eca-9f18-32f100a16c3c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.465737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 441/886] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-299f4974-266e-484d-bfdc-54ff3ad11e17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:29.466720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:29.467616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:29.468707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:29.468707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:29.468707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:29.468707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:29.468707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 446/887] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:29.473732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a59235-558b-4eca-9f18-32f100a16c3c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.474166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471619-c596-4390-a6b4-6dfe1fe3bac4 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.474450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471619-c596-4390-a6b4-6dfe1fe3bac4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.474721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16471619-c596-4390-a6b4-6dfe1fe3bac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.500484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-16471619-c596-4390-a6b4-6dfe1fe3bac4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:29.502211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-16471619-c596-4390-a6b4-6dfe1fe3bac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:29.585887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:29.592228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 447/888] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:29.607556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a59235-558b-4eca-9f18-32f100a16c3c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:29.619716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16471619-c596-4390-a6b4-6dfe1fe3bac4 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.639898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:29.644694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:29.645358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.645695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.645971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.646850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:29.647107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:29.648620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:29.662154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 442/889] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:29.669159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.746107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16471619-c596-4390-a6b4-6dfe1fe3bac4 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:29.783099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851e344-e2d0-40fb-8b40-a09d4c87e821 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:29.783462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:29.784177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.784457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.784728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.785510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:29.787061 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:29.799986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:29.801122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:29.801972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:29.807433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 448/890] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:29.808434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.822606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:29.823455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.823742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.824022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.832085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:29.833523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:29.850128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:29.852173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:29.853522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 443/891] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:06:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:29.860232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.876740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:29.919909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f92902-b235-4fd3-af8c-5689f6fc5878 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:29.934966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:29.955309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:29.955978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:29.956260 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:29.956533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:29.957321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:29.957538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:29.959049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:29.968761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 449/892] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:29.969633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:29.994577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343143a-e93c-4140-a475-c60b1ac0e530 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.038266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.039181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.039792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.040744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.042214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:30.042966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.045488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.071802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 444/893] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 11 11:06:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:30.073872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:30.088625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ecd65-64a2-4f7b-9309-5db59eeb5e31 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.130605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:30.139987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.147930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.147930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.147930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.147930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:30.147930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.147930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.154479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 450/894] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:30.155701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:30.181428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc728159-9726-4758-b4a9-c2f39bc1f7f8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.192817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:30.218187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a70b09-6df8-46a8-831f-43dac0a31f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.218845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70b09-6df8-46a8-831f-43dac0a31f2d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.219130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70b09-6df8-46a8-831f-43dac0a31f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.219438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70b09-6df8-46a8-831f-43dac0a31f2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.264225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227a8a0-f905-4b96-918e-cdbc7568a9e0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.289827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a70b09-6df8-46a8-831f-43dac0a31f2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.291233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a70b09-6df8-46a8-831f-43dac0a31f2d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.301197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.302078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.302552 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.302976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.303993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] RBAC: Adding query filter params (interface=public, service_id=4f750b6590694a76a365b4ef8c12c5c5, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:30.304455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.306029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.316332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 451/895] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 11 11:06:30 2022] GET /identity/v3/endpoints?service_id=4f750b6590694a76a365b4ef8c12c5c5&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:30.317399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:30.376232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:30.400564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 445/896] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:30 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:06:30.401622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a70b09-6df8-46a8-831f-43dac0a31f2d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:30.413735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:30.431468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a110c2-cbf7-4ca6-a0f2-9e26d7b5ea06 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.468364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8adde-146b-466b-ba77-6761aac498b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.469545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8adde-146b-466b-ba77-6761aac498b0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.470029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8adde-146b-466b-ba77-6761aac498b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.470526 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8adde-146b-466b-ba77-6761aac498b0 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.471447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8adde-146b-466b-ba77-6761aac498b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.473237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8adde-146b-466b-ba77-6761aac498b0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.487763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 452/897] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:06:30 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:06:30.510287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a70b09-6df8-46a8-831f-43dac0a31f2d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.517397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272d1d-6252-4abd-901e-7622763c5cd3 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.518045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272d1d-6252-4abd-901e-7622763c5cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.518533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04272d1d-6252-4abd-901e-7622763c5cd3 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.520497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 453/898] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:30.532045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d5a58-b098-4004-8745-d6b32e5d465e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.533143 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d5a58-b098-4004-8745-d6b32e5d465e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.533650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d5a58-b098-4004-8745-d6b32e5d465e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.552736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.553468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.553749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.554024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.558838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-771d5a58-b098-4004-8745-d6b32e5d465e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:30.560425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-771d5a58-b098-4004-8745-d6b32e5d465e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:30.566792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.568305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:30.589319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:30.590317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:30.591227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:30.591227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:30.591227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:30.591227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:30.591227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 446/899] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:30.596734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:30.655048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 454/900] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:30.667522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:30.679083 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d5a58-b098-4004-8745-d6b32e5d465e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:30.715750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb61bc5-2695-4de1-9a55-d7dfd1a9bbc0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.733535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:30.762251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723419a4-cd09-420e-b8e1-6f8577248927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.762923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723419a4-cd09-420e-b8e1-6f8577248927 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.763711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723419a4-cd09-420e-b8e1-6f8577248927 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.771140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723419a4-cd09-420e-b8e1-6f8577248927 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.771140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723419a4-cd09-420e-b8e1-6f8577248927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:30.771140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723419a4-cd09-420e-b8e1-6f8577248927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.771140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723419a4-cd09-420e-b8e1-6f8577248927 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.783300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 447/901] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:30.783300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaaf87-0a97-4aa3-a859-de27bf48c294 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.783300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaaf87-0a97-4aa3-a859-de27bf48c294 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.783300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceaaf87-0a97-4aa3-a859-de27bf48c294 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.783300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 448/902] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:30.784650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaaf87-0a97-4aa3-a859-de27bf48c294 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:30.794657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d5a58-b098-4004-8745-d6b32e5d465e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.830640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093f2b6-3679-49af-8feb-aab612a8b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.831229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f2b6-3679-49af-8feb-aab612a8b256 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.831544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f2b6-3679-49af-8feb-aab612a8b256 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.831824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093f2b6-3679-49af-8feb-aab612a8b256 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.832553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093f2b6-3679-49af-8feb-aab612a8b256 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.833920 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093f2b6-3679-49af-8feb-aab612a8b256 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.891204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceaaf87-0a97-4aa3-a859-de27bf48c294 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:30.905145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 455/903] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:06:30 2022] POST /identity/v3/services => generated 241 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:30.906603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bced1-54d7-4230-b3b2-7d35f64b5acb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.906904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bced1-54d7-4230-b3b2-7d35f64b5acb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.907175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bced1-54d7-4230-b3b2-7d35f64b5acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.926425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1b2bced1-54d7-4230-b3b2-7d35f64b5acb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:30.927820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1b2bced1-54d7-4230-b3b2-7d35f64b5acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:30.928391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:30.955059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:30.956636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:30.957164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:30.958186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:30.969633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:30.971771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:30.991280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:30.992257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:30.993211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:30.993211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:30.993211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:30.993211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:30.993211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:30.993211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:30.993211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:30.993211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:31.001018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 449/904] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:31.002251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:31.015683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 456/905] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:31.038933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bced1-54d7-4230-b3b2-7d35f64b5acb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:31.082691 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:31.114543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0f805-27ac-4d82-abd7-1571ca5882a7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:31.153931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bced1-54d7-4230-b3b2-7d35f64b5acb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:31.158357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:31.159027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:31.159341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:31.159617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:31.160469 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:31.160778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:31.162246 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:31.179061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 450/906] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:31.185759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:31.199671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:31.200809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:31.201382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:31.201657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:31.202326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:31.204293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:31.220223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:31.221408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:31.222460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:31.223465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 457/907] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 11 11:06:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:31.227745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:31.236099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:31.301874 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fe037-c7a1-4696-a0dc-84c5c265dab0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:31.336225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4409f3-3d93-4969-9af8-33d9dceeade6 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:31.338275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:31.348246 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:31.349218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:31.349643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:31.351409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:31.351759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] RBAC: Adding query filter params (role.id=131f3f505887436bb2a6499d9ead35c2, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=9d71050d3f36438d8b2d2cd4c537d3c2) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:31.352330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:31.354560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:31.367681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 451/908] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:31 2022] GET /identity/v3/role_assignments?user.id=9d71050d3f36438d8b2d2cd4c537d3c2&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=131f3f505887436bb2a6499d9ead35c2 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:31.373592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:31.374275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:31.374559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:31.374832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:31.375648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:31.375957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:31.377455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:31.383710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 458/909] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:31.389219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:31.505597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38caa8e2-c0c1-47b3-9902-18fd3b01cad3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:31.507645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:31.543309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:31.544294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:31.544752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:31.545201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:31.546099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:31.546593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:31.548143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:31.553812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 452/910] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:31.559541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:31.666275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b4aaf-b36f-407f-92d1-918639923d72 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:31.703131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd775-02c0-4732-8b38-e9f5af440ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:31.703825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd775-02c0-4732-8b38-e9f5af440ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:31.704107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd775-02c0-4732-8b38-e9f5af440ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:31.704411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd775-02c0-4732-8b38-e9f5af440ef1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:31.705238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd775-02c0-4732-8b38-e9f5af440ef1 None admin] RBAC: Adding query filter params (interface=public, service_id=89b1de8b91e74663b29b22f32a7b71da, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:31.705552 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd775-02c0-4732-8b38-e9f5af440ef1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:31.706909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd775-02c0-4732-8b38-e9f5af440ef1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:31.712801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 459/911] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 11 11:06:31 2022] GET /identity/v3/endpoints?service_id=89b1de8b91e74663b29b22f32a7b71da&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:31.718052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:31.818675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f67e91-686f-4be1-83fb-5c18049b622b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:31.853214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9249d-7efa-4e66-b462-e5941a47a420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:31.853884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9249d-7efa-4e66-b462-e5941a47a420 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:31.854228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9249d-7efa-4e66-b462-e5941a47a420 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:31.854515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9249d-7efa-4e66-b462-e5941a47a420 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:31.855195 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9249d-7efa-4e66-b462-e5941a47a420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:31.856592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9249d-7efa-4e66-b462-e5941a47a420 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:31.871022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 453/912] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:06:31 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:06:32.613500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912254de-774e-49c8-8311-0a4d31694720 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:32.614067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912254de-774e-49c8-8311-0a4d31694720 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:32.614067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912254de-774e-49c8-8311-0a4d31694720 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:32.615726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 460/913] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:32.620800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdff04-c909-40c3-8dd5-36fbd1ef26b2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:32.621221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdff04-c909-40c3-8dd5-36fbd1ef26b2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:32.621403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fdff04-c909-40c3-8dd5-36fbd1ef26b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:32.638353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-92fdff04-c909-40c3-8dd5-36fbd1ef26b2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:32.639572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-92fdff04-c909-40c3-8dd5-36fbd1ef26b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:32.668353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:32.711159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 454/914] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:32.733585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912254de-774e-49c8-8311-0a4d31694720 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:32.765681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:32.840699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912254de-774e-49c8-8311-0a4d31694720 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:32.876670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:32.877442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:32.877726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:32.878003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:32.878651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:32.880040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:32.892040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:32.893091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:32.893973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 461/915] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:32.904438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdff04-c909-40c3-8dd5-36fbd1ef26b2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.005002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fdff04-c909-40c3-8dd5-36fbd1ef26b2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:33.042522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eca983-ccdd-4ee7-bb39-6bcfdfd30f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:33.043190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eca983-ccdd-4ee7-bb39-6bcfdfd30f9e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.043509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eca983-ccdd-4ee7-bb39-6bcfdfd30f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.043786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eca983-ccdd-4ee7-bb39-6bcfdfd30f9e None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.044499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eca983-ccdd-4ee7-bb39-6bcfdfd30f9e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:33.044808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eca983-ccdd-4ee7-bb39-6bcfdfd30f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:33.046293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eca983-ccdd-4ee7-bb39-6bcfdfd30f9e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:33.051849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 455/916] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:33.057328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.066115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f18682-ecf5-4ce3-a6ad-e1ee8fd158ca None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.066401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f18682-ecf5-4ce3-a6ad-e1ee8fd158ca None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.066678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f18682-ecf5-4ce3-a6ad-e1ee8fd158ca None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.068302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 456/917] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:33.074544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f7142-7b39-499e-a2ca-d63f77a5f21b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.074544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f7142-7b39-499e-a2ca-d63f77a5f21b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.074705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f7142-7b39-499e-a2ca-d63f77a5f21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.091973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5f4f7142-7b39-499e-a2ca-d63f77a5f21b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:33.093156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5f4f7142-7b39-499e-a2ca-d63f77a5f21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:33.119380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:33.172166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:33.180224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1953-e7c5-4f20-ba9e-c27b3368a979 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:33.191377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 457/918] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:33.213784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f7142-7b39-499e-a2ca-d63f77a5f21b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.230807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:33.231517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.231800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.232071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.232779 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:33.233342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:33.235094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:33.241802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 462/919] 10.222.0.26 () {56 vars in 1114 bytes} [Mon Jul 11 11:06:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:33.247145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.327542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f7142-7b39-499e-a2ca-d63f77a5f21b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:33.364450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:33.365526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.365962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.366451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.367322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:33.368931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:33.373203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65a075-1ca2-484d-8bd5-8c83970eb050 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:33.381580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:33.382624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:33.383528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:33.386997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 458/920] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:33.392853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.413694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:33.421680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d62d8-3cae-46ec-8430-2a11d931f8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:33.422400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d62d8-3cae-46ec-8430-2a11d931f8f2 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.422863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d62d8-3cae-46ec-8430-2a11d931f8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.423150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d62d8-3cae-46ec-8430-2a11d931f8f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.424229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d62d8-3cae-46ec-8430-2a11d931f8f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:33.426101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d62d8-3cae-46ec-8430-2a11d931f8f2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:33.500836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80937d31-ead0-4973-a772-64060621a0d8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:33.515027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 463/921] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:06:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:06:33.516982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06850-b31a-4772-bc88-e96a7e88422b None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.517432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06850-b31a-4772-bc88-e96a7e88422b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.517711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad06850-b31a-4772-bc88-e96a7e88422b None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.519369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 464/922] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:33.526493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148d7a-8bc5-4e79-9c03-0abc0c4b8c58 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.526931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148d7a-8bc5-4e79-9c03-0abc0c4b8c58 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.527369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9148d7a-8bc5-4e79-9c03-0abc0c4b8c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.552665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b9148d7a-8bc5-4e79-9c03-0abc0c4b8c58 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:33.554040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b9148d7a-8bc5-4e79-9c03-0abc0c4b8c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:33.556808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:33.557725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.558214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.558653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.559550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:33.559994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:33.561585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:33.567921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 459/923] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:33.573819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:33.574113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.661226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 465/924] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:33.679128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:33.681253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148d7a-8bc5-4e79-9c03-0abc0c4b8c58 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.701266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61554c49-406f-46a8-95a2-b7f147dfcc79 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:33.741651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:33.742589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.743612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.744033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.744914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:33.745390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:33.746899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:33.752715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 460/925] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:33.759345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.804137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9148d7a-8bc5-4e79-9c03-0abc0c4b8c58 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:33.851730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e808-eac2-47de-b768-4e8f67ef4932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:33.853086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e808-eac2-47de-b768-4e8f67ef4932 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.853629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e808-eac2-47de-b768-4e8f67ef4932 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.853910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e808-eac2-47de-b768-4e8f67ef4932 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.869606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1e808-eac2-47de-b768-4e8f67ef4932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:33.870381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b3bb-73dc-4e3c-9194-dee5f9818835 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-7db1e808-eac2-47de-b768-4e8f67ef4932 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:33.881670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:33.882694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:33.884231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 466/926] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 11 11:06:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:33.890728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e808-eac2-47de-b768-4e8f67ef4932 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.895786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:33.921351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:33.922368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:33.922965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:33.923423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:33.924991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] RBAC: Adding query filter params (interface=public, service_id=0aa5e41710da472faec4111a3ad2fa42, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:33.925728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:33.927771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:33.939897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 461/927] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 11 11:06:33 2022] GET /identity/v3/endpoints?service_id=0aa5e41710da472faec4111a3ad2fa42&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:33.947196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:33.958437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:34.010807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e808-eac2-47de-b768-4e8f67ef4932 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.044515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabfa42-dcfe-4d44-b0ca-7b7fa54a2500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.045182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabfa42-dcfe-4d44-b0ca-7b7fa54a2500 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.045464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabfa42-dcfe-4d44-b0ca-7b7fa54a2500 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.045736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabfa42-dcfe-4d44-b0ca-7b7fa54a2500 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.046480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabfa42-dcfe-4d44-b0ca-7b7fa54a2500 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:34.046785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabfa42-dcfe-4d44-b0ca-7b7fa54a2500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.048172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabfa42-dcfe-4d44-b0ca-7b7fa54a2500 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.059410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 467/928] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 11 11:06:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:34.060714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32d253-f7b5-4440-9557-aed7ce8763cc None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.061002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32d253-f7b5-4440-9557-aed7ce8763cc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.061280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32d253-f7b5-4440-9557-aed7ce8763cc None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.062645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 468/929] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:34.070493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39a384-193f-4913-83a5-198fc0fc3378 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.070493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39a384-193f-4913-83a5-198fc0fc3378 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.070493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39a384-193f-4913-83a5-198fc0fc3378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.070791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d025760-0f13-4556-8ec3-ab49393a78f5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.090594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cc39a384-193f-4913-83a5-198fc0fc3378 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:34.091810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cc39a384-193f-4913-83a5-198fc0fc3378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:34.107990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26039b-81c8-4897-8786-928b7e609b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.108665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26039b-81c8-4897-8786-928b7e609b8c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.108957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26039b-81c8-4897-8786-928b7e609b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.109234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26039b-81c8-4897-8786-928b7e609b8c None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.109922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26039b-81c8-4897-8786-928b7e609b8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.111343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26039b-81c8-4897-8786-928b7e609b8c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.117417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 462/930] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:06:33 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:06:34.118252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26039b-81c8-4897-8786-928b7e609b8c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:34.178314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:34.187096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 469/931] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:34.212996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39a384-193f-4913-83a5-198fc0fc3378 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:34.215129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:34.257043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26039b-81c8-4897-8786-928b7e609b8c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.303397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.304213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.304501 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.304787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.325435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.326903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.349162 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39a384-193f-4913-83a5-198fc0fc3378 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:34.351609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:34.352553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:34.353474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:34.353474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:34.353474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:34.353474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:34.353474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 463/932] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:34.358495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:34.385784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.386696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.386977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.387260 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.387946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.389514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:34.402284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:34.404484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:34.406486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 470/933] 10.222.0.26 () {56 vars in 1132 bytes} [Mon Jul 11 11:06:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:34.411278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:34.477377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba4e2e-3ea9-487a-9c36-ca37671a52ab None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.507073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:34.511687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:34.523103 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.523693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979e1cf-f6df-4269-aca3-0480efcf53da None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.524221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.524680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.525158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.526918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:34.527490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.529017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.545651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 464/934] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:34.553976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:34.563358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.564028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.564311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.564586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.565825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:34.566136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.567537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.579241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 471/935] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:34.581722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:34.674230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2bf6e-c6e0-4f13-86d8-c948b2e8af88 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.711213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.712095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.712384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.712662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.718092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181ade3-e829-43d2-8edd-fad6f4ed317b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.721140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.723433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:34.742303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:34.743876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:34.744827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:34.747331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 465/936] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:06:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:34.750242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:34.752167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:34.771752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.773158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.773812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.774315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.775206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:34.776240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.782853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.786229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 472/937] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:34.792580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:34.801615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:34.882511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b06bc-aa6c-46de-bc78-c00b27627ac7 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.900506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9c6aa-79fb-49d2-afd3-75780218d68e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:34.929787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.930520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.930809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.931090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.931964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:34.932277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.933700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.940040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e5b0f-1d48-40f7-aab4-dfad7d612e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:34.940958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5b0f-1d48-40f7-aab4-dfad7d612e25 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:34.941246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5b0f-1d48-40f7-aab4-dfad7d612e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:34.941522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e5b0f-1d48-40f7-aab4-dfad7d612e25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:34.942511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e5b0f-1d48-40f7-aab4-dfad7d612e25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:34.944774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e5b0f-1d48-40f7-aab4-dfad7d612e25 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:34.955408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 466/938] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 11 11:06:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:34.959273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:35.043497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 473/939] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:06:34 2022] POST /identity/v3/endpoints => generated 347 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:35.067482 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:35.078401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8549f-5414-4dd1-bd86-ca57b4c5c9cc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:35.123370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:35.124723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:35.125607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:35.125886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:35.138265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:35.139844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:35.159124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:35.160127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:35.161033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:35.161033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:35.161033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:35.161033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:35.164905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 467/940] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:35.166824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e5b0f-1d48-40f7-aab4-dfad7d612e25 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:35.220812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:35.280347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e5b0f-1d48-40f7-aab4-dfad7d612e25 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:35.317800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:35.318423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:35.318712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:35.318989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:35.319749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:35.320120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:35.322009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:35.338980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 474/941] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:35.340347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:35.450584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3b40c-280c-4b6a-b4ae-75e0834e73d1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:35.486003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:35.487169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:35.487689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:35.488164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:35.495441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:35.497091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:35.511288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:35.512169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:35.513065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:35.513065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:35.513065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:35.513065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:35.513065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:35.513065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:35.513065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:35.513065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:35.515136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 468/942] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:35.527282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:35.629691 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b7f97-dc3a-4e60-bb71-28c4525fc899 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:35.631969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:35.664446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b7f95-ba02-4995-afdd-83361a9e1ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:35.665630 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b7f95-ba02-4995-afdd-83361a9e1ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:35.665914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b7f95-ba02-4995-afdd-83361a9e1ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:35.666188 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b7f95-ba02-4995-afdd-83361a9e1ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:35.666980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b7f95-ba02-4995-afdd-83361a9e1ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:35.667315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b7f95-ba02-4995-afdd-83361a9e1ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:35.668743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b7f95-ba02-4995-afdd-83361a9e1ad7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:35.686357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 475/943] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:35.687742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:35.699892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:35.708219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897cb6e-81a3-47b8-b1f3-939722d2988a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:35.708719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897cb6e-81a3-47b8-b1f3-939722d2988a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:35.708925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897cb6e-81a3-47b8-b1f3-939722d2988a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:35.710224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 476/944] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:35.716283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ce14-67ce-4ff5-affe-801aee110c6d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:35.716636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ce14-67ce-4ff5-affe-801aee110c6d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:35.716908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ce14-67ce-4ff5-affe-801aee110c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:35.734369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0668ce14-67ce-4ff5-affe-801aee110c6d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:35.735637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0668ce14-67ce-4ff5-affe-801aee110c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:35.812545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 477/945] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:35.814862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96101b37-73c0-49b4-92cf-f9c762100089 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:35.831729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668ce14-67ce-4ff5-affe-801aee110c6d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:35.852786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ae0ce-8162-4340-bf23-4a2d6326a5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:35.853583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ae0ce-8162-4340-bf23-4a2d6326a5af None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:35.853868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ae0ce-8162-4340-bf23-4a2d6326a5af None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:35.854143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ae0ce-8162-4340-bf23-4a2d6326a5af None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/9d71050d3f36438d8b2d2cd4c537d3c2/roles/131f3f505887436bb2a6499d9ead35c2` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:35.856483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ae0ce-8162-4340-bf23-4a2d6326a5af None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=9d71050d3f36438d8b2d2cd4c537d3c2, role_id=131f3f505887436bb2a6499d9ead35c2)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:35.858024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ae0ce-8162-4340-bf23-4a2d6326a5af None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:35.946680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 469/946] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:06:35 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/9d71050d3f36438d8b2d2cd4c537d3c2/roles/131f3f505887436bb2a6499d9ead35c2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:06:35.951290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:35.955464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668ce14-67ce-4ff5-affe-801aee110c6d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:36.000876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:36.001551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:36.001834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:36.002113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:36.002854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:36.004394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:36.005805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING keystone.server.flask.common [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 11 11:06:36.083532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.sql.core [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 11:06:36.083532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 11 11:06:36.083532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [parameters: {'id': '576b952fa9324ca29e9b757eabaf54b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 11 11:06:36.083532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80499) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 576b952fa9324ca29e9b757eabaf54b2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 576b952fa9324ca29e9b757eabaf54b2. Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:06:36.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result.read() Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 11 11:06:36.092079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application yield resource Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise self.value Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application session.commit() Jul 11 11:06:36.093550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.session.flush() Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application compat.raise_( Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 11:06:36.094460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 11:06:36.095396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result.read() Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:06:36.096222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [parameters: {'id': '576b952fa9324ca29e9b757eabaf54b2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:36.097135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:36.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 576b952fa9324ca29e9b757eabaf54b2. Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:36.100438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 478/947] 10.222.0.26 () {60 vars in 1140 bytes} [Mon Jul 11 11:06:35 2022] POST /identity/v3/projects => generated 262 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:36.108800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ae0ce-8162-4340-bf23-4a2d6326a5af None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:36.185979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:36.221197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ae0ce-8162-4340-bf23-4a2d6326a5af None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:36.257698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e4a3b-c299-4db5-bc42-00d11d6c3f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:36.260014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e4a3b-c299-4db5-bc42-00d11d6c3f41 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:36.260521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e4a3b-c299-4db5-bc42-00d11d6c3f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:36.261001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e4a3b-c299-4db5-bc42-00d11d6c3f41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:36.269351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e4a3b-c299-4db5-bc42-00d11d6c3f41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:36.271241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5e4a3b-c299-4db5-bc42-00d11d6c3f41 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5c5e4a3b-c299-4db5-bc42-00d11d6c3f41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:36.287147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:36.288283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:36.289405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:36.289405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:36.289405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:36.289405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:36.289405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:36.289405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:36.289405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:36.289405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:36.296657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 470/948] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:36.298369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:36.312383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b11526-1d37-40cc-898f-81bef5bb92ee None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:36.312699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b11526-1d37-40cc-898f-81bef5bb92ee None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:36.312941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b11526-1d37-40cc-898f-81bef5bb92ee None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:36.314642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 471/949] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:36.319479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c8a4a-19a5-40cc-86c5-8f35d42443ed None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:36.319765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c8a4a-19a5-40cc-86c5-8f35d42443ed None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:36.320044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c8a4a-19a5-40cc-86c5-8f35d42443ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:36.338224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7c8c8a4a-19a5-40cc-86c5-8f35d42443ed None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:36.339457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7c8c8a4a-19a5-40cc-86c5-8f35d42443ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:36.411637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 472/950] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:36.428563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410018-1c83-400f-913a-00cb13df94f5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:36.436279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c8a4a-19a5-40cc-86c5-8f35d42443ed None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:36.475641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:36.476632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:36.477142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:36.477683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:36.479137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:36.479633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:36.482288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:36.502365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:36.511495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 479/951] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:36.554633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c8a4a-19a5-40cc-86c5-8f35d42443ed None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:36.592439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:36.593261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:36.593541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:36.593810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:36.594468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:36.595873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:36.608668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:36.609870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:36.611424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 473/952] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:36.616366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:36.632750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:36.735079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73ed41-6973-406d-9f66-7145aae24d8e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:36.772423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:36.773335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:36.773787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:36.774234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:36.775111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:36.775599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:36.777116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:36.783040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 480/953] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:36.788717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:36.895893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2cc8ba-25e4-4e1f-9a79-7ec02165d303 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:36.918837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:36.931741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a7037-9fb7-4f33-b8c3-94d40debdef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:36.932790 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7037-9fb7-4f33-b8c3-94d40debdef9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:36.933221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7037-9fb7-4f33-b8c3-94d40debdef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:36.933645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7037-9fb7-4f33-b8c3-94d40debdef9 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:36.934496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a7037-9fb7-4f33-b8c3-94d40debdef9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:36.934963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a7037-9fb7-4f33-b8c3-94d40debdef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:36.936511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a7037-9fb7-4f33-b8c3-94d40debdef9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:36.941961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 474/954] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:36.948329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:37.036060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:37.062347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8208008-9bb8-4fca-a2f5-1ab4c0c31f56 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:37.097569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112750cc-69fc-4b9c-8d33-95619a4164be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:37.098226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112750cc-69fc-4b9c-8d33-95619a4164be None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:37.098505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112750cc-69fc-4b9c-8d33-95619a4164be None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:37.098777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112750cc-69fc-4b9c-8d33-95619a4164be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:37.099543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112750cc-69fc-4b9c-8d33-95619a4164be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:37.100908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112750cc-69fc-4b9c-8d33-95619a4164be None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:37.185743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 481/955] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:06:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:37.241882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20d7a-7ee7-4649-85fd-fe8f30eb146a None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:37.242247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20d7a-7ee7-4649-85fd-fe8f30eb146a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:37.242453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20d7a-7ee7-4649-85fd-fe8f30eb146a None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:37.244285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 475/956] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:37.250075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5f8cd-e3a1-4c92-9e93-c5412d874e60 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:37.250370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5f8cd-e3a1-4c92-9e93-c5412d874e60 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:37.250648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5f8cd-e3a1-4c92-9e93-c5412d874e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:37.268561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-13d5f8cd-e3a1-4c92-9e93-c5412d874e60 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:37.270133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-13d5f8cd-e3a1-4c92-9e93-c5412d874e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:37.349713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 482/957] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:37.372943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a20d7a-7ee7-4649-85fd-fe8f30eb146a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:37.478883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a20d7a-7ee7-4649-85fd-fe8f30eb146a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:37.513546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:37.514341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:37.514626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:37.514899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:37.515575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:37.516935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:37.529303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:37.530265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:37.531125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:37.534597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 476/958] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:37.536790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5f8cd-e3a1-4c92-9e93-c5412d874e60 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:37.594031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:37.639439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5f8cd-e3a1-4c92-9e93-c5412d874e60 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:37.675538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04ab8b-8e87-4e4c-875d-841be630089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:37.676211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04ab8b-8e87-4e4c-875d-841be630089b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:37.676492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04ab8b-8e87-4e4c-875d-841be630089b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:37.676768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04ab8b-8e87-4e4c-875d-841be630089b None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:37.677535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04ab8b-8e87-4e4c-875d-841be630089b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:37.677849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04ab8b-8e87-4e4c-875d-841be630089b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:37.679219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04ab8b-8e87-4e4c-875d-841be630089b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:37.690450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 483/959] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:37.692519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:37.744236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:37.796992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25321c0f-7973-4784-9780-f663c22ff5c2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:37.835645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8105dc0-d17d-4bc5-b8a2-86a2eec503f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:37.836331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8105dc0-d17d-4bc5-b8a2-86a2eec503f9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:37.836653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8105dc0-d17d-4bc5-b8a2-86a2eec503f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:37.836931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8105dc0-d17d-4bc5-b8a2-86a2eec503f9 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:37.837652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8105dc0-d17d-4bc5-b8a2-86a2eec503f9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:37.837958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8105dc0-d17d-4bc5-b8a2-86a2eec503f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:37.839357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8105dc0-d17d-4bc5-b8a2-86a2eec503f9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:37.848896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 477/960] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:38.098748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e012751-2e76-435c-964b-e58c89197f29 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:38.099381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e012751-2e76-435c-964b-e58c89197f29 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:38.099381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e012751-2e76-435c-964b-e58c89197f29 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:38.100949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 484/961] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:38.106537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6e079-a10b-4916-8b1b-7032937b0326 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:38.106962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6e079-a10b-4916-8b1b-7032937b0326 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:38.107158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6e079-a10b-4916-8b1b-7032937b0326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:38.124164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-90b6e079-a10b-4916-8b1b-7032937b0326 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:38.125419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-90b6e079-a10b-4916-8b1b-7032937b0326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:38.194575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 478/962] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:38.219024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e012751-2e76-435c-964b-e58c89197f29 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:38.338439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e012751-2e76-435c-964b-e58c89197f29 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:38.361255 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:38.378650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde9952-2e71-40fb-9f07-b08345b6d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:38.380858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9952-2e71-40fb-9f07-b08345b6d7ce None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:38.380858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9952-2e71-40fb-9f07-b08345b6d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:38.380858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9952-2e71-40fb-9f07-b08345b6d7ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:38.393112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde9952-2e71-40fb-9f07-b08345b6d7ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-afde9952-2e71-40fb-9f07-b08345b6d7ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:38.402314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:38.403332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:38.404198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 485/963] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 11 11:06:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:38.411342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6e079-a10b-4916-8b1b-7032937b0326 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:38.464795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:38.520954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6e079-a10b-4916-8b1b-7032937b0326 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:38.564058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:38.564745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:38.565031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:38.565311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:38.566074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:38.566385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:38.567792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:38.576694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 479/964] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 11 11:06:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:38.582144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde9952-2e71-40fb-9f07-b08345b6d7ce None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:38.690070 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde9952-2e71-40fb-9f07-b08345b6d7ce None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:38.728525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42da107-9eb6-40a5-b451-3af9045ec261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:38.729319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42da107-9eb6-40a5-b451-3af9045ec261 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:38.729603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42da107-9eb6-40a5-b451-3af9045ec261 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:38.729880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42da107-9eb6-40a5-b451-3af9045ec261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:38.741210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42da107-9eb6-40a5-b451-3af9045ec261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:38.742596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42da107-9eb6-40a5-b451-3af9045ec261 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f42da107-9eb6-40a5-b451-3af9045ec261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:38.761380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:38.762338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:38.763731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:38.763731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:38.763731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:38.763731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:38.763731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 486/965] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:38.772344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:38.879355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318abc4a-7456-4b86-beb9-42e09f3b90e3 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:38.879685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318abc4a-7456-4b86-beb9-42e09f3b90e3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:38.879916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318abc4a-7456-4b86-beb9-42e09f3b90e3 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:38.881437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 487/966] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:38.886838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65cf28-1c6b-4e94-a659-9169b61b6b9a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:38.887435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65cf28-1c6b-4e94-a659-9169b61b6b9a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:38.888045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65cf28-1c6b-4e94-a659-9169b61b6b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:38.898400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2e342-78ea-49ec-891b-a87c92b5dedd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:38.906138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:38.907966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2d65cf28-1c6b-4e94-a659-9169b61b6b9a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:38.909448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2d65cf28-1c6b-4e94-a659-9169b61b6b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:38.934127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:38.934787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:38.935165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:38.935400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:38.936106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:38.936422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:38.937799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:38.940399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:38.950952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 480/967] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:38.956253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:38.995367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 488/968] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:39.014722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65cf28-1c6b-4e94-a659-9169b61b6b9a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:39.058927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a59a2-2727-4d46-a242-89e967fd099a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:39.095640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:39.096177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.096522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.096867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.105770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:39.107344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:39.123101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:39.124187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:39.125593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 481/969] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:06:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:39.130184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:39.137718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65cf28-1c6b-4e94-a659-9169b61b6b9a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:39.157555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:39.177434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:39.178859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.179363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.180094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.181610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:39.183668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:39.230077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.sql.core [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 11:06:39.230077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 11 11:06:39.230077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [parameters: {'user_id': 'a9297efd51e342e79091d018b577e122', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 11 11:06:39.230077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80499) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 11:06:39.233557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0065c2b-ce43-475a-9989-d9962b001991 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result.read() Jul 11 11:06:39.240131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:06:39.241423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application yield resource Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application next(self.gen) Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise self.value Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application session.commit() Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 11:06:39.242629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.session.flush() Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application compat.raise_( Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 11:06:39.243660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 11:06:39.244584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application conn.query(q) Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application result.read() Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 11:06:39.245439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a9297efd51e342e79091d018b577e122', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:39.252937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 11 11:06:39.255026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 11 11:06:39.257198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 11 11:06:39.257198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 11 11:06:39.257198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 11:06:39.257198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 11:06:39.257198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 11 11:06:39.257198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:39.257198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 489/970] 10.222.0.26 () {60 vars in 1131 bytes} [Mon Jul 11 11:06:39 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:39.257198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:39.286672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:39.300692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:39.301692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.302197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.303177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.304205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:39.305029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:39.307764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:39.333147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 482/971] 10.222.0.26 () {56 vars in 1149 bytes} [Mon Jul 11 11:06:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:39.339126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:39.377259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26187618-c0ab-4851-b5c1-e621f20f46b0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:39.417217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43c3b2-ea3e-4b1e-be39-8c0c6dc32c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:39.418206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c3b2-ea3e-4b1e-be39-8c0c6dc32c44 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.418494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c3b2-ea3e-4b1e-be39-8c0c6dc32c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.418767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c3b2-ea3e-4b1e-be39-8c0c6dc32c44 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.426792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43c3b2-ea3e-4b1e-be39-8c0c6dc32c44 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:39.428449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43c3b2-ea3e-4b1e-be39-8c0c6dc32c44 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:39.435276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1a43c3b2-ea3e-4b1e-be39-8c0c6dc32c44 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:39.444050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:39.450994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 490/972] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc0a8b-9582-4370-ae8a-fc5229dc99a5 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.451983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc0a8b-9582-4370-ae8a-fc5229dc99a5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.452696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc0a8b-9582-4370-ae8a-fc5229dc99a5 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.452696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 491/973] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:39.452696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc0a8b-9582-4370-ae8a-fc5229dc99a5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:39.460361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f2af4-a268-4b60-93ba-ddb822bcb042 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:39.498956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91966c84-7b94-42df-885b-4958d120e1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:39.499560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966c84-7b94-42df-885b-4958d120e1de None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.499842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966c84-7b94-42df-885b-4958d120e1de None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.500120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91966c84-7b94-42df-885b-4958d120e1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.542563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:39.562208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc0a8b-9582-4370-ae8a-fc5229dc99a5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:39.573420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91966c84-7b94-42df-885b-4958d120e1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:39.575065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91966c84-7b94-42df-885b-4958d120e1de None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:39.598850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85352d84-35a9-4a63-ac60-f5b2e416b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:39.599543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352d84-35a9-4a63-ac60-f5b2e416b48c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.599827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352d84-35a9-4a63-ac60-f5b2e416b48c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.600101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85352d84-35a9-4a63-ac60-f5b2e416b48c None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.600904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85352d84-35a9-4a63-ac60-f5b2e416b48c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:39.601210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85352d84-35a9-4a63-ac60-f5b2e416b48c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:39.603186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85352d84-35a9-4a63-ac60-f5b2e416b48c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:39.639311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 492/974] 10.222.0.26 () {56 vars in 1101 bytes} [Mon Jul 11 11:06:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:39.639311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7d4d-799b-4c73-bb6e-349496442dee None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.639311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7d4d-799b-4c73-bb6e-349496442dee None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.639311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e7d4d-799b-4c73-bb6e-349496442dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.665429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0c5e7d4d-799b-4c73-bb6e-349496442dee None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:39.668343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0c5e7d4d-799b-4c73-bb6e-349496442dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:39.677610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:39.697665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 483/975] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:39 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:06:39.703710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91966c84-7b94-42df-885b-4958d120e1de None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:39.773475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 493/976] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:39.798120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e7d4d-799b-4c73-bb6e-349496442dee None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:39.849575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91966c84-7b94-42df-885b-4958d120e1de None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:39.895996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:39.896798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.897076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.897352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.898741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:39.910150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:39.911644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:39.935162 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e7d4d-799b-4c73-bb6e-349496442dee None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:39.937310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:39.938522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:39.940045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:39.940045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:39.940045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:39.940045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:39.940045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 484/977] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:39.944230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:39.980211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:39.982926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:39.983769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:39.984055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:39.984331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:39.984992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:39.986484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:39.999374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:40.000428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:40.001617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 494/978] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:40.006662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:40.050552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4161946-99c4-4572-aca7-d149b73f8f21 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.085245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b8b0b-26d2-4242-9126-c53afdf68784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.085902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b8b0b-26d2-4242-9126-c53afdf68784 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.086183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b8b0b-26d2-4242-9126-c53afdf68784 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.086502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b8b0b-26d2-4242-9126-c53afdf68784 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.087224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b8b0b-26d2-4242-9126-c53afdf68784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:40.087560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b8b0b-26d2-4242-9126-c53afdf68784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.088910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b8b0b-26d2-4242-9126-c53afdf68784 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.099839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 485/979] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:40.101087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6ac26-e09c-41e4-93ba-5e8c6f6cf431 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.101387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6ac26-e09c-41e4-93ba-5e8c6f6cf431 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.101671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6ac26-e09c-41e4-93ba-5e8c6f6cf431 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.103619 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 486/980] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:40.107965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6ac26-e09c-41e4-93ba-5e8c6f6cf431 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:40.118478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c5719-663d-4fe4-897c-10e5b6ff0f95 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.152393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e3ab0-2d60-4f33-abb1-7fc3a0f48a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.153045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e3ab0-2d60-4f33-abb1-7fc3a0f48a4d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.153323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e3ab0-2d60-4f33-abb1-7fc3a0f48a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.153595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e3ab0-2d60-4f33-abb1-7fc3a0f48a4d None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.154290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e3ab0-2d60-4f33-abb1-7fc3a0f48a4d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:40.154596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e3ab0-2d60-4f33-abb1-7fc3a0f48a4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.155977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e3ab0-2d60-4f33-abb1-7fc3a0f48a4d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.157837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:40.161467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 495/981] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:40.162663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae3fd1-81f4-484e-86fa-656465a29323 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.162947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae3fd1-81f4-484e-86fa-656465a29323 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.163220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae3fd1-81f4-484e-86fa-656465a29323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.179353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1eae3fd1-81f4-484e-86fa-656465a29323 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:40.181090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1eae3fd1-81f4-484e-86fa-656465a29323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:40.206725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:40.213544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6ac26-e09c-41e4-93ba-5e8c6f6cf431 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.254183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 496/982] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:40.254183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae3fd1-81f4-484e-86fa-656465a29323 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:40.254500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.254500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.254500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.254500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.260295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.261737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:40.275820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:40.276734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:40.277648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 487/983] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:40.278154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:40.345883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae3fd1-81f4-484e-86fa-656465a29323 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.374171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:40.374828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67471d-1e35-42e5-8424-611a22e50f48 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.380081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.380839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.381121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.381399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.382098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:40.382402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.383821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.393480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 497/984] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:40.393480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:40.407630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c50d0-5f2e-473b-ac12-f87d0c64e026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.408296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c50d0-5f2e-473b-ac12-f87d0c64e026 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.408578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c50d0-5f2e-473b-ac12-f87d0c64e026 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.408857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c50d0-5f2e-473b-ac12-f87d0c64e026 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.409578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c50d0-5f2e-473b-ac12-f87d0c64e026 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.410926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c50d0-5f2e-473b-ac12-f87d0c64e026 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.486934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee70b86-291b-49d3-af08-eef3f6450f4c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.495020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 488/985] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:06:40 2022] POST /identity/v3/services => generated 242 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:40.496055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c50d0-5f2e-473b-ac12-f87d0c64e026 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:40.517264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:40.539036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.540497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.540615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.540804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.541592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:40.541900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.543425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.558640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 498/986] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:40.565534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:40.608814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c50d0-5f2e-473b-ac12-f87d0c64e026 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.622521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:40.646628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.646628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.646628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.646628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.647412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] RBAC: Adding query filter params (interface=admin, service_id=0aa5e41710da472faec4111a3ad2fa42, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:40.647731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.649293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.657434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 489/987] 10.222.0.26 () {56 vars in 1249 bytes} [Mon Jul 11 11:06:40 2022] GET /identity/v3/endpoints?service_id=0aa5e41710da472faec4111a3ad2fa42&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:40.663191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:40.680184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f46155-f959-4ca4-96e7-e3ad755a074c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.717477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be60fb-dd7f-4c00-b6d4-d162aad70638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.718129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be60fb-dd7f-4c00-b6d4-d162aad70638 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.718415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be60fb-dd7f-4c00-b6d4-d162aad70638 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.718692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be60fb-dd7f-4c00-b6d4-d162aad70638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.719553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be60fb-dd7f-4c00-b6d4-d162aad70638 None admin] RBAC: Adding query filter params (role.id=131f3f505887436bb2a6499d9ead35c2, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=9d71050d3f36438d8b2d2cd4c537d3c2) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:40.719866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be60fb-dd7f-4c00-b6d4-d162aad70638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.721288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be60fb-dd7f-4c00-b6d4-d162aad70638 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.735488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 499/988] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:40 2022] GET /identity/v3/role_assignments?user.id=9d71050d3f36438d8b2d2cd4c537d3c2&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=131f3f505887436bb2a6499d9ead35c2 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:40.753369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:40.775550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e9ba7-8f32-4638-ac21-08f4923b9552 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:40.813685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c848f4c-f49b-4904-b3ac-3e6c2c640fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:40.814379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c848f4c-f49b-4904-b3ac-3e6c2c640fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:40.814662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c848f4c-f49b-4904-b3ac-3e6c2c640fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:40.814937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c848f4c-f49b-4904-b3ac-3e6c2c640fa3 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:40.815664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c848f4c-f49b-4904-b3ac-3e6c2c640fa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:40.817041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c848f4c-f49b-4904-b3ac-3e6c2c640fa3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:40.827350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 490/989] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:06:40 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:06:42.550577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e1f4d-f32c-413f-af2e-10e790907afa None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:42.551136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e1f4d-f32c-413f-af2e-10e790907afa None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:42.551136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e1f4d-f32c-413f-af2e-10e790907afa None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:42.552663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 500/990] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:42.557610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be319d-31f9-4632-8da0-8219d9dbd18b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:42.558035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be319d-31f9-4632-8da0-8219d9dbd18b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:42.558202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be319d-31f9-4632-8da0-8219d9dbd18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:42.574810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-01be319d-31f9-4632-8da0-8219d9dbd18b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:42.576075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-01be319d-31f9-4632-8da0-8219d9dbd18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:42.643767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 491/991] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:42.667545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e1f4d-f32c-413f-af2e-10e790907afa None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:42.767660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e1f4d-f32c-413f-af2e-10e790907afa None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:42.770221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1c75-3f76-4a24-8990-bea7f2ffe642 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:42.770520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1c75-3f76-4a24-8990-bea7f2ffe642 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:42.770802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1c75-3f76-4a24-8990-bea7f2ffe642 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:42.772392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 492/992] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:42.777695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824426-a40e-4c15-8bca-b6569706d017 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:42.777987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824426-a40e-4c15-8bca-b6569706d017 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:42.778269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824426-a40e-4c15-8bca-b6569706d017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:42.783663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:42.796621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-92824426-a40e-4c15-8bca-b6569706d017 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:42.798748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-92824426-a40e-4c15-8bca-b6569706d017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:42.808644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:42.809372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:42.809684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:42.809965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:42.810655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:42.812049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:42.825395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:42.826706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:42.827771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 501/993] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:42.832375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:42.877906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 493/994] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:42.879389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e977b-5326-47ef-ab22-19a5f60c1e4f None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:42.879680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e977b-5326-47ef-ab22-19a5f60c1e4f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:42.879957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e977b-5326-47ef-ab22-19a5f60c1e4f None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:42.881319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 494/995] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:42.886891 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae92fdb-0168-4e66-8297-9931c4354a16 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:42.887182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae92fdb-0168-4e66-8297-9931c4354a16 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:42.887493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae92fdb-0168-4e66-8297-9931c4354a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:42.893998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:42.905403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1ae92fdb-0168-4e66-8297-9931c4354a16 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:42.906586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ae92fdb-0168-4e66-8297-9931c4354a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:42.935565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c893490-a500-4113-a552-d24782a6a24c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:42.968686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:42.969338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:42.969622 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:42.969896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:42.970599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] RBAC: Adding query filter params (name=compute) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:42.970906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:42.972301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:42.972977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 495/996] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:42.973804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae92fdb-0168-4e66-8297-9931c4354a16 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:42.977730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 502/997] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:42.982992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.026764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:43.079270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae92fdb-0168-4e66-8297-9931c4354a16 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.087438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4dedb-911e-41f1-becd-ad4e8ce519ce None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.113720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.114500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.114898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.115045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] PATH_INFO: `/v3/services/image` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.115716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.117065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.118901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:43.120391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.121049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.121332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.121605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.122322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:43.122624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.123990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.128747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 503/998] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:43.128949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:43.129933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:43.130840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:43.131648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.131723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 496/999] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 11 11:06:42 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:43.134874 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.230763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4129c-26a5-4d2b-ab50-e9a77854f5f8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.233381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe413-9fd4-46e7-8144-a453fdbf5e97 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.238331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:43.265397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.266047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.266165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.266333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.266610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.266851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.267135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.267423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] RBAC: Adding query filter params (interface=public, service_id=0c70e6202a2446c4916a2f0ae83beebc, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:43.267537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.267746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.268083 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.269098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.269425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.274599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 497/1000] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 11 11:06:43 2022] GET /identity/v3/endpoints?service_id=0c70e6202a2446c4916a2f0ae83beebc&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:43.275396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:43.281803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:43.282839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:43.283740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 504/1001] 10.222.0.26 () {56 vars in 1126 bytes} [Mon Jul 11 11:06:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:43.284572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.339614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:43.377563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec654-d679-478a-81c1-f6a640d2d387 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.383686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6d85e-4168-4f38-915d-ce8711afdd3e None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.410132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.410795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.411074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.411374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.412077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:43.412385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.413737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.416240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319b2de-01e5-4aa3-aca8-a055c7c009b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.416882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b2de-01e5-4aa3-aca8-a055c7c009b3 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.417158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b2de-01e5-4aa3-aca8-a055c7c009b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.417427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319b2de-01e5-4aa3-aca8-a055c7c009b3 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.418103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319b2de-01e5-4aa3-aca8-a055c7c009b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.418415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 498/1002] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:43.419184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.419546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319b2de-01e5-4aa3-aca8-a055c7c009b3 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.443297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 505/1003] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:06:43 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:06:43.443297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319b2de-01e5-4aa3-aca8-a055c7c009b3 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.470496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:43.529900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a75e2e-1270-4a1a-9e37-32a16f593ab3 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.554742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319b2de-01e5-4aa3-aca8-a055c7c009b3 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.560084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:43.572601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.573686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.573973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.574245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.574959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:43.575284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.577279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.583696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 499/1004] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:43.589821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.592955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.594659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.594948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.595232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.596000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:43.596314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.597693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.607118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 506/1005] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:43 2022] GET /identity/v3/services?type=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:43.702352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc183c-0114-462f-8839-77234a69c3e6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.738001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef7281-87c7-4dfc-99a6-fe0fbe3fdbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.738684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7281-87c7-4dfc-99a6-fe0fbe3fdbee None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.738971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7281-87c7-4dfc-99a6-fe0fbe3fdbee None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.739264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef7281-87c7-4dfc-99a6-fe0fbe3fdbee None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.739997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7281-87c7-4dfc-99a6-fe0fbe3fdbee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:43.740308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7281-87c7-4dfc-99a6-fe0fbe3fdbee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.741690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef7281-87c7-4dfc-99a6-fe0fbe3fdbee None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.747408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 500/1006] 10.222.0.26 () {56 vars in 1122 bytes} [Mon Jul 11 11:06:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:43.753161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:43.854569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:43.858874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc5aff-fbc5-4d01-acac-2dc87685c7a7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:43.892779 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0646486-bb99-43c6-aa27-fefbcb565876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:43.893434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646486-bb99-43c6-aa27-fefbcb565876 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:43.893715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646486-bb99-43c6-aa27-fefbcb565876 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:43.893990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0646486-bb99-43c6-aa27-fefbcb565876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:43.894737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0646486-bb99-43c6-aa27-fefbcb565876 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:43.896168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0646486-bb99-43c6-aa27-fefbcb565876 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:43.946650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 507/1007] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:06:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:45.470235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883c495-38a2-46e1-8073-c7a6c061941c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:45.470771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883c495-38a2-46e1-8073-c7a6c061941c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:45.470771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883c495-38a2-46e1-8073-c7a6c061941c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:45.472360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 501/1008] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:45.477767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1c9c-85a7-4f6f-b97b-f2c895cec93d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:45.478230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1c9c-85a7-4f6f-b97b-f2c895cec93d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:45.478313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c1c9c-85a7-4f6f-b97b-f2c895cec93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:45.495370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-212c1c9c-85a7-4f6f-b97b-f2c895cec93d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:45.496536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-212c1c9c-85a7-4f6f-b97b-f2c895cec93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:45.571287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 508/1009] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:45 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:45.592385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883c495-38a2-46e1-8073-c7a6c061941c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:45.624511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675691ba-b362-4895-bec0-0af0d4ceb663 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:45.624810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675691ba-b362-4895-bec0-0af0d4ceb663 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:45.625074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675691ba-b362-4895-bec0-0af0d4ceb663 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:45.626445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 509/1010] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:45.631217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb12ea-7c87-47a5-a6ec-3a048021a743 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:45.631527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb12ea-7c87-47a5-a6ec-3a048021a743 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:45.631803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb12ea-7c87-47a5-a6ec-3a048021a743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:45.648777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-befb12ea-7c87-47a5-a6ec-3a048021a743 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:45.649944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-befb12ea-7c87-47a5-a6ec-3a048021a743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:45.655679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:45.705496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883c495-38a2-46e1-8073-c7a6c061941c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:45.721628 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 510/1011] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:45 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:45.741595 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:45.742427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:45.742708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:45.743053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:45.743756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:45.744342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb12ea-7c87-47a5-a6ec-3a048021a743 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:45.745121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:45.757026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:45.759296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:45.760188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 502/1012] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:45.762774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:45.764275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:45.892051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2c6fc-de3b-4b54-a644-736a2fbee359 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:45.894722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb12ea-7c87-47a5-a6ec-3a048021a743 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:45.911071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:45.928661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:45.929371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:45.929660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:45.929962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:45.930684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:45.931004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:45.932408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:45.933183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ccce2-fa14-43d8-9068-01a35dcef64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:45.933902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ccce2-fa14-43d8-9068-01a35dcef64b None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:45.934183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ccce2-fa14-43d8-9068-01a35dcef64b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:45.934561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ccce2-fa14-43d8-9068-01a35dcef64b None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:45.935190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ccce2-fa14-43d8-9068-01a35dcef64b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:45.936570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ccce2-fa14-43d8-9068-01a35dcef64b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:45.937786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 503/1013] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:45.943320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:46.011080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 511/1014] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:06:45 2022] POST /identity/v3/services => generated 242 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:06:46.012436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cccd4-71e3-49e0-92cc-22106c42e39d None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:46.012729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cccd4-71e3-49e0-92cc-22106c42e39d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:46.013337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cccd4-71e3-49e0-92cc-22106c42e39d None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:46.015289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 512/1015] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:46.021403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e34a-cdc7-4da3-acc0-83de17b4cc8c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:46.022048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e34a-cdc7-4da3-acc0-83de17b4cc8c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:46.022509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0e34a-cdc7-4da3-acc0-83de17b4cc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:46.042610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-49d0e34a-cdc7-4da3-acc0-83de17b4cc8c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:46.049069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-49d0e34a-cdc7-4da3-acc0-83de17b4cc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:46.061991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f0782-9802-403d-8d45-ac44693a1520 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:46.108158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:46.108837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:46.109119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:46.109397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:46.110111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:46.110424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:46.111845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:46.117281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 504/1016] 10.222.0.26 () {56 vars in 1112 bytes} [Mon Jul 11 11:06:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:46.123271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:46.143279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 513/1017] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:46 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:46.157587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0e34a-cdc7-4da3-acc0-83de17b4cc8c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:46.161573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:46.169676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:46.230360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ee58c-0a84-4da0-96cf-b0518293a0b6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:46.266787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eff28-490e-4e17-8631-fa377964b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:46.267726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eff28-490e-4e17-8631-fa377964b004 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:46.268014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eff28-490e-4e17-8631-fa377964b004 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:46.268294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eff28-490e-4e17-8631-fa377964b004 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:46.269043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3eff28-490e-4e17-8631-fa377964b004 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:46.270410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3eff28-490e-4e17-8631-fa377964b004 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:46.272445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0e34a-cdc7-4da3-acc0-83de17b4cc8c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:46.308104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45b935-b4a0-49ed-a272-adfd1099bf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:46.308893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b935-b4a0-49ed-a272-adfd1099bf62 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:46.309180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b935-b4a0-49ed-a272-adfd1099bf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:46.309460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b935-b4a0-49ed-a272-adfd1099bf62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:46.310677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45b935-b4a0-49ed-a272-adfd1099bf62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:46.312136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45b935-b4a0-49ed-a272-adfd1099bf62 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:46.313761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 514/1018] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:46.319479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45b935-b4a0-49ed-a272-adfd1099bf62 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:46.347001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 505/1019] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:06:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:06:46.405656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:46.438920 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45b935-b4a0-49ed-a272-adfd1099bf62 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:46.481983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48246475-f062-4525-b31a-60000dd4b3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:46.482701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48246475-f062-4525-b31a-60000dd4b3fd None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:46.482976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48246475-f062-4525-b31a-60000dd4b3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:46.483204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48246475-f062-4525-b31a-60000dd4b3fd None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:46.483949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48246475-f062-4525-b31a-60000dd4b3fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:46.485406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48246475-f062-4525-b31a-60000dd4b3fd None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:46.581847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 515/1020] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:06:46 2022] POST /identity/v3/projects => generated 311 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:06:48.052862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e29d3-3f96-4f5a-a79d-c430fa9b3cb6 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.052862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e29d3-3f96-4f5a-a79d-c430fa9b3cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.052862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e29d3-3f96-4f5a-a79d-c430fa9b3cb6 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.052862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 506/1021] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:48.057912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3422a0-b80e-4e95-bf30-507cd953a9a2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.058333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3422a0-b80e-4e95-bf30-507cd953a9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.058512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3422a0-b80e-4e95-bf30-507cd953a9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.077526 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-af3422a0-b80e-4e95-bf30-507cd953a9a2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:48.078802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-af3422a0-b80e-4e95-bf30-507cd953a9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:48.158129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 516/1022] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:48.180930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e29d3-3f96-4f5a-a79d-c430fa9b3cb6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:48.277663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:48.282801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e29d3-3f96-4f5a-a79d-c430fa9b3cb6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:48.316975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66152152-2f5f-4efa-8f1e-28ad7af5f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:48.316975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152152-2f5f-4efa-8f1e-28ad7af5f113 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.316975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152152-2f5f-4efa-8f1e-28ad7af5f113 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.316975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66152152-2f5f-4efa-8f1e-28ad7af5f113 None admin] PATH_INFO: `/v3/services/image` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.317378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66152152-2f5f-4efa-8f1e-28ad7af5f113 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:48.318653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66152152-2f5f-4efa-8f1e-28ad7af5f113 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-66152152-2f5f-4efa-8f1e-28ad7af5f113 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:48.330418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:48.331744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:48.332647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:48.335580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 507/1023] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 11 11:06:48 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:48.338074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3422a0-b80e-4e95-bf30-507cd953a9a2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:48.387286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:48.436436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3422a0-b80e-4e95-bf30-507cd953a9a2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:48.471468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:48.472122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.472402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.472673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.473376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:48.473680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:48.475026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:48.481185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883fc4f-096f-4e92-ab28-6fa09538492e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.481738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883fc4f-096f-4e92-ab28-6fa09538492e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.481858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 517/1024] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:48.481953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883fc4f-096f-4e92-ab28-6fa09538492e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.483209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 508/1025] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:48.485304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:48.487722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e1ac-9c23-49bc-869b-d0842689fadb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.488014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e1ac-9c23-49bc-869b-d0842689fadb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.488291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863e1ac-9c23-49bc-869b-d0842689fadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.507360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e863e1ac-9c23-49bc-869b-d0842689fadb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:48.509092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e863e1ac-9c23-49bc-869b-d0842689fadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:48.579174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 509/1026] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:48.584371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53ab4e-71f2-4fe1-9c82-d330bed950f5 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:48.598327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:48.602272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863e1ac-9c23-49bc-869b-d0842689fadb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:48.620548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:48.621196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.621481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.621757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.622735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] RBAC: Adding query filter params (type=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:48.623223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:48.624932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:48.630092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 518/1027] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:48.636291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:48.695203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:48.704524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863e1ac-9c23-49bc-869b-d0842689fadb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:48.732684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e2540-d94a-4d5d-b683-d2c2d9593adf None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:48.737650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6409fc-df4e-4a8a-8006-ba9252efd6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:48.738460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6409fc-df4e-4a8a-8006-ba9252efd6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.738742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6409fc-df4e-4a8a-8006-ba9252efd6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.739017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6409fc-df4e-4a8a-8006-ba9252efd6b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.749609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6409fc-df4e-4a8a-8006-ba9252efd6b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cf6409fc-df4e-4a8a-8006-ba9252efd6b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:48.758347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:48.759632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:48.760506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 510/1028] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 11 11:06:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:48.761228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60578bb6-db20-4b30-8daf-0cadee944721 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.761228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60578bb6-db20-4b30-8daf-0cadee944721 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.761515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60578bb6-db20-4b30-8daf-0cadee944721 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.762850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 511/1029] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:48.767592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60578bb6-db20-4b30-8daf-0cadee944721 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:48.770285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1373a-01f7-4ac7-b331-4ce0c945ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:48.770945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1373a-01f7-4ac7-b331-4ce0c945ad4b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.771233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1373a-01f7-4ac7-b331-4ce0c945ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.771540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1373a-01f7-4ac7-b331-4ce0c945ad4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.772338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1373a-01f7-4ac7-b331-4ce0c945ad4b None admin] RBAC: Adding query filter params (interface=public, service_id=5058c5cea3ad4be1bf2313dc7be4bce5, region_id=RegionOne) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:48.772687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1373a-01f7-4ac7-b331-4ce0c945ad4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:48.774087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1373a-01f7-4ac7-b331-4ce0c945ad4b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:48.779195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 519/1030] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 11 11:06:48 2022] GET /identity/v3/endpoints?service_id=5058c5cea3ad4be1bf2313dc7be4bce5&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:48.780591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b4d6-ad19-428e-8f1b-4e6f2a0de0b1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.780878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b4d6-ad19-428e-8f1b-4e6f2a0de0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.781155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b4d6-ad19-428e-8f1b-4e6f2a0de0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.797549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-dd90b4d6-ad19-428e-8f1b-4e6f2a0de0b1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:48.798993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-dd90b4d6-ad19-428e-8f1b-4e6f2a0de0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:48.834907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:48.863773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 520/1031] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:48.864598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90b4d6-ad19-428e-8f1b-4e6f2a0de0b1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:48.874459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60578bb6-db20-4b30-8daf-0cadee944721 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:48.908833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:48.909505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.909867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.910144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.910898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:48.911208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:48.912593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:48.921972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 512/1032] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 11 11:06:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:48.922749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:48.940070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:48.962432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90b4d6-ad19-428e-8f1b-4e6f2a0de0b1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:48.996117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cecc73-ac0e-469a-86fc-1f5ac51defb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:48.996775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cecc73-ac0e-469a-86fc-1f5ac51defb4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:48.997057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cecc73-ac0e-469a-86fc-1f5ac51defb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:48.997334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cecc73-ac0e-469a-86fc-1f5ac51defb4 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:48.998019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cecc73-ac0e-469a-86fc-1f5ac51defb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:48.999409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cecc73-ac0e-469a-86fc-1f5ac51defb4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:49.005940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 521/1033] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:06:48 2022] GET /identity/v3/services => generated 1957 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:06:49.006712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cecc73-ac0e-469a-86fc-1f5ac51defb4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:49.023882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496f5f-503d-4ea9-8e39-f252e4912020 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:49.045888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:49.077286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62362a4-4dfd-41ad-acd1-894e7af5e597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:49.078314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62362a4-4dfd-41ad-acd1-894e7af5e597 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:49.078602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62362a4-4dfd-41ad-acd1-894e7af5e597 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:49.078880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62362a4-4dfd-41ad-acd1-894e7af5e597 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:49.080168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62362a4-4dfd-41ad-acd1-894e7af5e597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:49.082387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62362a4-4dfd-41ad-acd1-894e7af5e597 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:49.085051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 513/1034] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:49.090813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62362a4-4dfd-41ad-acd1-894e7af5e597 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:49.121960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cecc73-ac0e-469a-86fc-1f5ac51defb4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:49.160655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:49.161436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:49.161724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:49.161991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:49.174563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:49.176039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:49.183563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:49.195631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:49.196580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:49.197510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:49.197510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:49.197510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:49.197510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:49.197510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 522/1035] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:49.202632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:49.204861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62362a4-4dfd-41ad-acd1-894e7af5e597 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:49.243175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8586c2-7123-4217-affe-bd75566e5435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:49.243881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8586c2-7123-4217-affe-bd75566e5435 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:49.244166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8586c2-7123-4217-affe-bd75566e5435 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:49.244443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8586c2-7123-4217-affe-bd75566e5435 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:49.245159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8586c2-7123-4217-affe-bd75566e5435 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:49.246613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8586c2-7123-4217-affe-bd75566e5435 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:49.295075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:49.306828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934e7a1-f08a-4660-855d-544c630592d8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:49.340984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50771a05-8761-4385-9108-f87c090da631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:49.341878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771a05-8761-4385-9108-f87c090da631 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:49.342305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771a05-8761-4385-9108-f87c090da631 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:49.342720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50771a05-8761-4385-9108-f87c090da631 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:49.343613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50771a05-8761-4385-9108-f87c090da631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:49.344059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50771a05-8761-4385-9108-f87c090da631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:49.346945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 514/1036] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:06:49 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:06:49.347074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50771a05-8761-4385-9108-f87c090da631 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:49.358817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 523/1037] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:49.364785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8586c2-7123-4217-affe-bd75566e5435 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:49.472685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8586c2-7123-4217-affe-bd75566e5435 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:49.509248 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:49.510047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:49.510329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:49.510601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:49.518440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:49.519903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:49.521693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:49.534429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:49.536999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:49.538030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 515/1038] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:06:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:49.541637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50771a05-8761-4385-9108-f87c090da631 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:49.644268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50771a05-8761-4385-9108-f87c090da631 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:49.677777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:49.678447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:49.678727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:49.679000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:49.679886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:49.680196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:49.681610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:49.699412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 524/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:06:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:49.707170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:49.808999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ea53-0ade-403a-9602-573fc860eb97 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:49.853294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b0db4-7d18-422d-b931-f02f25e4fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:49.853982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b0db4-7d18-422d-b931-f02f25e4fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:49.854269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b0db4-7d18-422d-b931-f02f25e4fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:49.854549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b0db4-7d18-422d-b931-f02f25e4fbe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:49.919872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:49.924117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b0db4-7d18-422d-b931-f02f25e4fbe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:49.925590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b0db4-7d18-422d-b931-f02f25e4fbe8 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:50.031586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 516/1040] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:06:50.033223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:50.043167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:50.147667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c654d4-0175-4b22-ab9f-b6995f078301 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:50.184199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:50.184922 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:50.185215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:50.185481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:50.199666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:50.201131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:50.221305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:50.222264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:50.223184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:50.223184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:50.223184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:50.223184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:50.227398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 525/1041] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:50.228397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b0db4-7d18-422d-b931-f02f25e4fbe8 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:50.322980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:50.327283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b0db4-7d18-422d-b931-f02f25e4fbe8 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:50.361065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:50.361723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:50.362009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:50.362343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:50.363067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:50.363409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:50.364775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:50.375917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 517/1042] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:50.381379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:50.439303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:50.480849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157db50-9355-45b5-af30-322184bfa7e0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:50.515153 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:50.515981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:50.516267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:50.516543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:50.523331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:50.524779 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:50.539742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:50.540752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:50.541671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:50.541671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:50.541671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:50.541671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:50.541671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:50.541671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:50.541671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:50.541671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:50.543609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 526/1043] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:50.547057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:50.648007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f39b2-b004-49cc-ba07-6357a0488ed0 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:50.682149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d72a6-0dc8-4a5b-bd16-822e8a513a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:50.682819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d72a6-0dc8-4a5b-bd16-822e8a513a43 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:50.683104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d72a6-0dc8-4a5b-bd16-822e8a513a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:50.683411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d72a6-0dc8-4a5b-bd16-822e8a513a43 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:50.684196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d72a6-0dc8-4a5b-bd16-822e8a513a43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:50.684510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d72a6-0dc8-4a5b-bd16-822e8a513a43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:50.685925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d72a6-0dc8-4a5b-bd16-822e8a513a43 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:50.698085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 518/1044] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:50.703898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:50.803848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf649eb-8005-4e20-a989-942d4fde5709 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:50.820723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:50.837913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf63f65-dd37-48af-84aa-7a6b8ef03b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:50.838566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf63f65-dd37-48af-84aa-7a6b8ef03b55 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:50.838856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf63f65-dd37-48af-84aa-7a6b8ef03b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:50.839136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf63f65-dd37-48af-84aa-7a6b8ef03b55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:50.840005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf63f65-dd37-48af-84aa-7a6b8ef03b55 None admin] RBAC: Adding query filter params (role.id=131f3f505887436bb2a6499d9ead35c2, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=258ce5f4d6ff400199a80157cc34f5e5) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:50.840320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf63f65-dd37-48af-84aa-7a6b8ef03b55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:50.841755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf63f65-dd37-48af-84aa-7a6b8ef03b55 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:50.847229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 527/1045] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:50 2022] GET /identity/v3/role_assignments?user.id=258ce5f4d6ff400199a80157cc34f5e5&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=131f3f505887436bb2a6499d9ead35c2 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:51.055033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55e00c-f912-46c5-a8c9-d20ed4c1bcbd None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.055362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55e00c-f912-46c5-a8c9-d20ed4c1bcbd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.055569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55e00c-f912-46c5-a8c9-d20ed4c1bcbd None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.057044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 519/1046] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:51.062125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96cda3-4eb3-4187-b4d9-a35349280c94 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.062413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96cda3-4eb3-4187-b4d9-a35349280c94 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.062689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96cda3-4eb3-4187-b4d9-a35349280c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.083828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3d96cda3-4eb3-4187-b4d9-a35349280c94 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:51.085022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3d96cda3-4eb3-4187-b4d9-a35349280c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:51.154244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 528/1047] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:51.177540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55e00c-f912-46c5-a8c9-d20ed4c1bcbd None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:51.197782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:51.287490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55e00c-f912-46c5-a8c9-d20ed4c1bcbd None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:51.323607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef11882-db26-49a4-9a82-2be3adcd25ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:51.324394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef11882-db26-49a4-9a82-2be3adcd25ed None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.324681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef11882-db26-49a4-9a82-2be3adcd25ed None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.324958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef11882-db26-49a4-9a82-2be3adcd25ed None admin] PATH_INFO: `/v3/services/image` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.325613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef11882-db26-49a4-9a82-2be3adcd25ed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:51.326974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef11882-db26-49a4-9a82-2be3adcd25ed None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7ef11882-db26-49a4-9a82-2be3adcd25ed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:51.339041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:51.340077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 11:06:51.340926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:51.343580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 520/1048] 10.222.0.26 () {56 vars in 1105 bytes} [Mon Jul 11 11:06:51 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:51.346256 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96cda3-4eb3-4187-b4d9-a35349280c94 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:51.447005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96cda3-4eb3-4187-b4d9-a35349280c94 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:51.458217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce810cad-0739-4814-b7ec-23b1efa58474 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.458878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce810cad-0739-4814-b7ec-23b1efa58474 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.459168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce810cad-0739-4814-b7ec-23b1efa58474 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.460865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 521/1049] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:51.461517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:51.465674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140dcfb-9f9c-4781-b61c-5cce42bb0751 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.465966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140dcfb-9f9c-4781-b61c-5cce42bb0751 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.466247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140dcfb-9f9c-4781-b61c-5cce42bb0751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.483477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:51.483938 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8140dcfb-9f9c-4781-b61c-5cce42bb0751 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:51.484139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.484421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.484697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.485128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8140dcfb-9f9c-4781-b61c-5cce42bb0751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:51.485406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] RBAC: Adding query filter params (name=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:51.485711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:51.487166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:51.493873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 529/1050] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:51.499287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:51.559507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 522/1051] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:51.582997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140dcfb-9f9c-4781-b61c-5cce42bb0751 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:51.597774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:51.611647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a9ae3-d03a-4e74-aece-de503c379b21 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:51.646385 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:51.647042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.647354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.647633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.648847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] RBAC: Adding query filter params (type=image) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:51.649156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:51.650515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:51.663949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 530/1052] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:51.665069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:51.686065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140dcfb-9f9c-4781-b61c-5cce42bb0751 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:51.709879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:51.723557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ca79-ff07-40f5-8d04-85e9e81b2616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:51.724782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ca79-ff07-40f5-8d04-85e9e81b2616 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.724782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ca79-ff07-40f5-8d04-85e9e81b2616 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.724961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ca79-ff07-40f5-8d04-85e9e81b2616 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.735074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02ca79-ff07-40f5-8d04-85e9e81b2616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6c02ca79-ff07-40f5-8d04-85e9e81b2616 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:51.743356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:51.744321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:51.745197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 523/1053] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:51.750374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ca79-ff07-40f5-8d04-85e9e81b2616 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:51.772110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b7d9d-bd4c-4f0e-af78-ff0ee297bc44 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:51.813390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79a52f-d5cb-425b-bc8d-775d6a0f64cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:51.813823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79a52f-d5cb-425b-bc8d-775d6a0f64cb None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.814104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79a52f-d5cb-425b-bc8d-775d6a0f64cb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.814478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79a52f-d5cb-425b-bc8d-775d6a0f64cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.815113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79a52f-d5cb-425b-bc8d-775d6a0f64cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:51.816549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79a52f-d5cb-425b-bc8d-775d6a0f64cb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:51.825856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:51.854807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ca79-ff07-40f5-8d04-85e9e81b2616 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:51.886451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:51.887120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:51.887432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:51.887708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:51.888448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:51.888754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:51.890102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:51.899915 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 524/1054] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:51.905213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:51.938967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 531/1055] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:06:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:06:52.019473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2a76e-65ea-43a2-acc3-362c11db0625 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:52.075789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:52.078524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd631-8a3d-4d44-9d81-20601c99d39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:52.079460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd631-8a3d-4d44-9d81-20601c99d39f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:52.079746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd631-8a3d-4d44-9d81-20601c99d39f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:52.080025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd631-8a3d-4d44-9d81-20601c99d39f None admin] PATH_INFO: `/v3/users/de8da6e3a9c34202abeeb7ec77387805` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:52.094777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bd631-8a3d-4d44-9d81-20601c99d39f None admin] RBAC: Authorizing `identity:get_user(user_id=de8da6e3a9c34202abeeb7ec77387805)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:52.096244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bd631-8a3d-4d44-9d81-20601c99d39f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:52.098272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 525/1056] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:06:51 2022] GET /identity/v3/users/de8da6e3a9c34202abeeb7ec77387805 => generated 285 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:52.104033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79a52f-d5cb-425b-bc8d-775d6a0f64cb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:52.209340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:52.215701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79a52f-d5cb-425b-bc8d-775d6a0f64cb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:52.248267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb848bf-315e-4869-bf3d-82a0a594cc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:52.249378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb848bf-315e-4869-bf3d-82a0a594cc7f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:52.250212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb848bf-315e-4869-bf3d-82a0a594cc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:52.250515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb848bf-315e-4869-bf3d-82a0a594cc7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:52.322830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb848bf-315e-4869-bf3d-82a0a594cc7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:52.324307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb848bf-315e-4869-bf3d-82a0a594cc7f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:52.414919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:52.426891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 532/1057] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:06:52.428349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd631-8a3d-4d44-9d81-20601c99d39f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:52.531320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd631-8a3d-4d44-9d81-20601c99d39f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:52.565224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75c1c8-dc05-4710-8bdb-cf301bc99f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:52.566080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75c1c8-dc05-4710-8bdb-cf301bc99f4a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:52.566363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75c1c8-dc05-4710-8bdb-cf301bc99f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:52.566639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75c1c8-dc05-4710-8bdb-cf301bc99f4a None admin] PATH_INFO: `/v3/projects/edc06fb0dec6448386d40635b9e8faba` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:52.567842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75c1c8-dc05-4710-8bdb-cf301bc99f4a None admin] RBAC: Authorizing `identity:get_project(project_id=edc06fb0dec6448386d40635b9e8faba)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:52.569263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75c1c8-dc05-4710-8bdb-cf301bc99f4a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:52.571144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 526/1058] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:06:52 2022] GET /identity/v3/projects/edc06fb0dec6448386d40635b9e8faba => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:52.576926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb848bf-315e-4869-bf3d-82a0a594cc7f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:52.674928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb848bf-315e-4869-bf3d-82a0a594cc7f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:52.708613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa4027-a257-4b8b-8dff-bb663f287966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:52.709267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4027-a257-4b8b-8dff-bb663f287966 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:52.709549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4027-a257-4b8b-8dff-bb663f287966 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:52.709825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4027-a257-4b8b-8dff-bb663f287966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:52.710683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa4027-a257-4b8b-8dff-bb663f287966 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=edc06fb0dec6448386d40635b9e8faba, user.id=de8da6e3a9c34202abeeb7ec77387805) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:52.710997 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa4027-a257-4b8b-8dff-bb663f287966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:52.712451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa4027-a257-4b8b-8dff-bb663f287966 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:52.723335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 533/1059] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:52 2022] GET /identity/v3/role_assignments?user.id=de8da6e3a9c34202abeeb7ec77387805&scope.project.id=edc06fb0dec6448386d40635b9e8faba&role.id=007af6bd1b734517b934b4cac45faf2c => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:52.887784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f1ab-4f78-4ba5-be80-3bd1cd20035a None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:52.888121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f1ab-4f78-4ba5-be80-3bd1cd20035a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:52.888316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2f1ab-4f78-4ba5-be80-3bd1cd20035a None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:52.889765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 527/1060] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:52.894877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef61d29-e6b1-47c7-9cfd-89b34782b576 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:52.895174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef61d29-e6b1-47c7-9cfd-89b34782b576 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:52.895501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef61d29-e6b1-47c7-9cfd-89b34782b576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:52.912062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-aef61d29-e6b1-47c7-9cfd-89b34782b576 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:52.913251 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-aef61d29-e6b1-47c7-9cfd-89b34782b576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:52.970099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:52.982428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 534/1061] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:06:53.005211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2f1ab-4f78-4ba5-be80-3bd1cd20035a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:53.073394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:53.119536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2f1ab-4f78-4ba5-be80-3bd1cd20035a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:53.153425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a54414-3c13-4fd5-97c6-f6c83227bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:53.154162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a54414-3c13-4fd5-97c6-f6c83227bd37 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:53.154449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a54414-3c13-4fd5-97c6-f6c83227bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:53.154727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a54414-3c13-4fd5-97c6-f6c83227bd37 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:53.165798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a54414-3c13-4fd5-97c6-f6c83227bd37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-29a54414-3c13-4fd5-97c6-f6c83227bd37 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:53.174216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:53.175180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:53.176435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:53.179351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 528/1062] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 11 11:06:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:53.181935 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef61d29-e6b1-47c7-9cfd-89b34782b576 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:53.278984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef61d29-e6b1-47c7-9cfd-89b34782b576 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:53.312587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:53.313254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:53.313535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:53.313814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:53.314564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:53.314875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:53.316264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:53.329667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 535/1063] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 11 11:06:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:53.331172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a54414-3c13-4fd5-97c6-f6c83227bd37 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:53.431738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a54414-3c13-4fd5-97c6-f6c83227bd37 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:53.448273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:53.466349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:53.467144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:53.467457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:53.467734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:53.478087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:53.479550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:53.497112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:53.498033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:53.498943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:53.498943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:53.498943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:53.498943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:53.503388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 529/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:53.504184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:53.557089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:53.604277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6891b01-90a0-41e2-a04b-41fabd906194 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:53.637336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:53.637998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:53.638280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:53.638554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:53.639294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:53.639608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:53.640963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:53.657774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 536/1065] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:53.659400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:53.761922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5de1d8-919c-496c-b5de-f90cec0d6bd4 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:53.796288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc40fa5-6de6-4916-b568-bda16afc91b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:53.797087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40fa5-6de6-4916-b568-bda16afc91b7 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:53.797374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40fa5-6de6-4916-b568-bda16afc91b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:53.797654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40fa5-6de6-4916-b568-bda16afc91b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:53.806590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc40fa5-6de6-4916-b568-bda16afc91b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:53.808105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc40fa5-6de6-4916-b568-bda16afc91b7 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6fc40fa5-6de6-4916-b568-bda16afc91b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:53.823207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:53.824274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:53.825286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:53.827577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 530/1066] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:06:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:53.830318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:53.930675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4429f-5f16-48c2-856b-3a14fe9e902b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:53.943528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:53.964353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:53.965018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:53.965297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:53.965571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:53.966414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:53.966706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:53.968152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:53.989443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 537/1067] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:06:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:53.989642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3cb84-4b21-4f64-827d-68a984d24f9c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:53.989642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3cb84-4b21-4f64-827d-68a984d24f9c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:53.989642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3cb84-4b21-4f64-827d-68a984d24f9c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:53.989642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 531/1068] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:53.995369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:53.995497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5759a9e-e536-49cc-bdc1-032aab59f8b1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:53.995497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5759a9e-e536-49cc-bdc1-032aab59f8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:53.995497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5759a9e-e536-49cc-bdc1-032aab59f8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.014745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e5759a9e-e536-49cc-bdc1-032aab59f8b1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:54.016154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e5759a9e-e536-49cc-bdc1-032aab59f8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:54.042333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:54.098042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 532/1069] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:54.104227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037239e-9d41-4d18-bff0-962a719b229c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:54.120639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5759a9e-e536-49cc-bdc1-032aab59f8b1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:54.143328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:54.144121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.144399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.144667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.155261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:54.156719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:54.174005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:54.175113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:54.176111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:54.176111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:54.176111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:54.176111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:54.176111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 538/1070] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:54.180824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:54.205586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:54.231453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5759a9e-e536-49cc-bdc1-032aab59f8b1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:54.266976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395e1b-1a09-4d0a-b38c-97d81f3aa46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:54.267961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395e1b-1a09-4d0a-b38c-97d81f3aa46a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.268544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395e1b-1a09-4d0a-b38c-97d81f3aa46a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.269014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395e1b-1a09-4d0a-b38c-97d81f3aa46a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.283745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d46e7-2411-4282-938a-1b4f627d7659 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:54.298790 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:54.322073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:54.322733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.323058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.323367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.324089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:54.324394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:54.325806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:54.338785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 539/1071] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:54.342399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395e1b-1a09-4d0a-b38c-97d81f3aa46a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:54.344024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395e1b-1a09-4d0a-b38c-97d81f3aa46a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:54.344134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:54.433149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:54.447101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 533/1072] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:06:54.452868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395e1b-1a09-4d0a-b38c-97d81f3aa46a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:54.456156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83870f-eb23-48a5-aa60-e120fc159c2b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:54.490368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:54.491164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.491472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.491750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.499132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:54.500593 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:54.513136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:54.514583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:54.515534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:54.516462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 540/1073] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:54.527197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:54.558719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395e1b-1a09-4d0a-b38c-97d81f3aa46a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:54.593496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:54.594290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.594577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.594862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.605646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:54.607190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:54.624691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:54.625610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:54.631868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:54.631868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:54.631868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:54.631868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:54.632109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471854-ab8b-4323-84cb-93ea2580a8a4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:54.632189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 534/1074] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:54.632189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:54.666507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:54.667159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.667468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.667746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.668532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:54.668840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:54.670247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:54.682965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 541/1075] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:54.688435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:54.691163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:54.739888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27dee6-5c4b-45ee-b882-0803c1bb1360 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:54.751242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:54.774035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed198d-66bc-4f0f-b93d-fc46d8e41a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:54.774694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed198d-66bc-4f0f-b93d-fc46d8e41a54 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.774976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed198d-66bc-4f0f-b93d-fc46d8e41a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.775266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed198d-66bc-4f0f-b93d-fc46d8e41a54 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.775983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed198d-66bc-4f0f-b93d-fc46d8e41a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:54.776289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed198d-66bc-4f0f-b93d-fc46d8e41a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:54.777683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed198d-66bc-4f0f-b93d-fc46d8e41a54 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:54.789546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 535/1076] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:54.789681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaad3c1-f62a-46e8-b0d0-eda41f7cf959 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:54.790974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b99e85-8942-40e9-9aa2-27967da2c84c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.791281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b99e85-8942-40e9-9aa2-27967da2c84c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.791564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b99e85-8942-40e9-9aa2-27967da2c84c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.792944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 536/1077] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:54.802547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143f231-97f4-45b4-a428-033591a64c29 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.802838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143f231-97f4-45b4-a428-033591a64c29 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.803304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143f231-97f4-45b4-a428-033591a64c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.822382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f143f231-97f4-45b4-a428-033591a64c29 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:54.823871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f143f231-97f4-45b4-a428-033591a64c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:54.830053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf2646-cad9-40ef-a450-8baf037d4e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:54.831353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf2646-cad9-40ef-a450-8baf037d4e1b None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:54.831821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf2646-cad9-40ef-a450-8baf037d4e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:54.832863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf2646-cad9-40ef-a450-8baf037d4e1b None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/258ce5f4d6ff400199a80157cc34f5e5/roles/131f3f505887436bb2a6499d9ead35c2` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:54.836272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf2646-cad9-40ef-a450-8baf037d4e1b None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=258ce5f4d6ff400199a80157cc34f5e5, role_id=131f3f505887436bb2a6499d9ead35c2)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:54.838513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf2646-cad9-40ef-a450-8baf037d4e1b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:54.895186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 537/1078] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:54.918630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143f231-97f4-45b4-a428-033591a64c29 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:54.929232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 542/1079] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:06:54 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/258ce5f4d6ff400199a80157cc34f5e5/roles/131f3f505887436bb2a6499d9ead35c2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:06:55.030878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:55.034909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143f231-97f4-45b4-a428-033591a64c29 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:55.082995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ec978-15fc-4868-b835-84dddb5b1f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:55.083836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ec978-15fc-4868-b835-84dddb5b1f06 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:55.084123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ec978-15fc-4868-b835-84dddb5b1f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:55.084400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ec978-15fc-4868-b835-84dddb5b1f06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:55.096432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ec978-15fc-4868-b835-84dddb5b1f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6e3ec978-15fc-4868-b835-84dddb5b1f06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:55.104124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:55.105171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:55.106214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 538/1080] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:55.111532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf2646-cad9-40ef-a450-8baf037d4e1b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:55.171457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:55.209620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf2646-cad9-40ef-a450-8baf037d4e1b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:55.243402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:55.244067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:55.244348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:55.244621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:55.245368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:55.245677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:55.247032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:55.255035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 543/1081] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:55.261024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ec978-15fc-4868-b835-84dddb5b1f06 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:55.358420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ec978-15fc-4868-b835-84dddb5b1f06 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:55.390380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d14c9f-0c89-413b-93b6-24db6d2c051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:55.391278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d14c9f-0c89-413b-93b6-24db6d2c051c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:55.391576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d14c9f-0c89-413b-93b6-24db6d2c051c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:55.391857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d14c9f-0c89-413b-93b6-24db6d2c051c None admin] PATH_INFO: `/v3/users/de8da6e3a9c34202abeeb7ec77387805` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:55.393053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d14c9f-0c89-413b-93b6-24db6d2c051c None admin] RBAC: Authorizing `identity:get_user(user_id=de8da6e3a9c34202abeeb7ec77387805)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:55.394486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d14c9f-0c89-413b-93b6-24db6d2c051c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:55.396142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 539/1082] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:06:55 2022] GET /identity/v3/users/de8da6e3a9c34202abeeb7ec77387805 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:55.402884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:55.499940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f519cf-9498-4977-ab3b-94e296fefff9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:55.521574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:55.532379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de856ba-9206-411d-bd43-7b20b32e91aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:55.540183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de856ba-9206-411d-bd43-7b20b32e91aa None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:55.540183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de856ba-9206-411d-bd43-7b20b32e91aa None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:55.540183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de856ba-9206-411d-bd43-7b20b32e91aa None admin] PATH_INFO: `/v3/projects/edc06fb0dec6448386d40635b9e8faba` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:55.540183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de856ba-9206-411d-bd43-7b20b32e91aa None admin] RBAC: Authorizing `identity:get_project(project_id=edc06fb0dec6448386d40635b9e8faba)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:55.540183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de856ba-9206-411d-bd43-7b20b32e91aa None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:55.540183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 544/1083] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:06:55 2022] GET /identity/v3/projects/edc06fb0dec6448386d40635b9e8faba => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:55.544312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d14c9f-0c89-413b-93b6-24db6d2c051c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:55.572873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:55.642234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d14c9f-0c89-413b-93b6-24db6d2c051c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:55.674616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108b388-ceae-44bc-8ac3-71a567a6b9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:55.675534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108b388-ceae-44bc-8ac3-71a567a6b9b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:55.675817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108b388-ceae-44bc-8ac3-71a567a6b9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:55.676091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108b388-ceae-44bc-8ac3-71a567a6b9b9 None admin] PATH_INFO: `/v3/projects/edc06fb0dec6448386d40635b9e8faba/users/de8da6e3a9c34202abeeb7ec77387805/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:55.678011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108b388-ceae-44bc-8ac3-71a567a6b9b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=edc06fb0dec6448386d40635b9e8faba, user_id=de8da6e3a9c34202abeeb7ec77387805, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:55.679573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108b388-ceae-44bc-8ac3-71a567a6b9b9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:55.745824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 540/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:06:55 2022] PUT /identity/v3/projects/edc06fb0dec6448386d40635b9e8faba/users/de8da6e3a9c34202abeeb7ec77387805/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:06:56.957189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96040-c83c-4458-a521-a551726b8fca None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:56.957835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96040-c83c-4458-a521-a551726b8fca None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:56.957937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96040-c83c-4458-a521-a551726b8fca None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:56.959236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 545/1085] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:06:56.964955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c95b26-707f-4446-bfbd-4ec933090dce None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:56.965366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c95b26-707f-4446-bfbd-4ec933090dce None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:56.965635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c95b26-707f-4446-bfbd-4ec933090dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:56.982353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a3c95b26-707f-4446-bfbd-4ec933090dce None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:56.983543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a3c95b26-707f-4446-bfbd-4ec933090dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:57.046207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 541/1086] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:57.070557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d96040-c83c-4458-a521-a551726b8fca None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:57.168614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d96040-c83c-4458-a521-a551726b8fca None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:57.201112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12816c2-9d4f-4e0f-9993-5ae467a2996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:57.201909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12816c2-9d4f-4e0f-9993-5ae467a2996f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.202189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12816c2-9d4f-4e0f-9993-5ae467a2996f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.202460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12816c2-9d4f-4e0f-9993-5ae467a2996f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.212863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12816c2-9d4f-4e0f-9993-5ae467a2996f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-d12816c2-9d4f-4e0f-9993-5ae467a2996f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:57.220750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:57.221908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 11:06:57.223016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:57.227477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 546/1087] 10.222.0.26 () {56 vars in 1120 bytes} [Mon Jul 11 11:06:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:57.228538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c95b26-707f-4446-bfbd-4ec933090dce None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:57.246842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:57.326815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c95b26-707f-4446-bfbd-4ec933090dce None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:57.359809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:57.360466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.360746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.361018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.361752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:57.362058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:57.363426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:57.371100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 542/1088] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 11 11:06:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:57.376744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12816c2-9d4f-4e0f-9993-5ae467a2996f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:57.427967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:57.476620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12816c2-9d4f-4e0f-9993-5ae467a2996f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:57.509904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:57.510687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.510974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.511265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.520894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:57.522332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:57.539058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:57.540148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:57.541076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:57.541076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:57.541076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:57.541076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:57.541076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 547/1089] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:57.546046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:57.624440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:57.642910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91958984-282d-4bbd-87cb-c12febbf802a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:57.675427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e3715-fde0-411d-b1fd-097334aa9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:57.676080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e3715-fde0-411d-b1fd-097334aa9920 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.676361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e3715-fde0-411d-b1fd-097334aa9920 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.676633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e3715-fde0-411d-b1fd-097334aa9920 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.677344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e3715-fde0-411d-b1fd-097334aa9920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:57.677651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e3715-fde0-411d-b1fd-097334aa9920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:57.678998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e3715-fde0-411d-b1fd-097334aa9920 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:57.690770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 543/1090] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:57.697381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:57.738894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9614f9-be4c-4505-9e73-d26da335e81b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.739100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9614f9-be4c-4505-9e73-d26da335e81b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.739410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9614f9-be4c-4505-9e73-d26da335e81b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.740777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 544/1091] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:06:57.746037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330957ef-0e41-4072-b5f7-c6ad93872f87 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.746324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330957ef-0e41-4072-b5f7-c6ad93872f87 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.746600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330957ef-0e41-4072-b5f7-c6ad93872f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.763235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-330957ef-0e41-4072-b5f7-c6ad93872f87 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:06:57.764430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-330957ef-0e41-4072-b5f7-c6ad93872f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:06:57.796517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977d046-11a4-4e61-88fa-5da897185900 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:57.799050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:57.829178 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:57.829957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.830239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.830510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.834743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 545/1092] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:06:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:06:57.838038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:57.839499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:06:57.854235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:57.855419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 11:06:57.856532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:57.857317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 548/1093] 10.222.0.26 () {56 vars in 1093 bytes} [Mon Jul 11 11:06:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:57.858567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330957ef-0e41-4072-b5f7-c6ad93872f87 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:57.861103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:57.925953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:57.956735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330957ef-0e41-4072-b5f7-c6ad93872f87 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:57.959241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6613-c52d-4922-8359-9d31876a9d2c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:57.990590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eabf3-9cf8-417c-ba94-42414ec77e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:57.991365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eabf3-9cf8-417c-ba94-42414ec77e37 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.991651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eabf3-9cf8-417c-ba94-42414ec77e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.992389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:57.992514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eabf3-9cf8-417c-ba94-42414ec77e37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.993047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:57.993339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:57.993612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:57.994449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:57.994759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:57.996215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.003300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9eabf3-9cf8-417c-ba94-42414ec77e37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cd9eabf3-9cf8-417c-ba94-42414ec77e37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:06:58.010890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:58.012085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:06:58.013214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 546/1094] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:06:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:06:58.016444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 549/1095] 10.222.0.26 () {56 vars in 1133 bytes} [Mon Jul 11 11:06:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:58.017939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eabf3-9cf8-417c-ba94-42414ec77e37 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.021797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.045390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.119598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eabf3-9cf8-417c-ba94-42414ec77e37 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.122807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daf054-cbde-477e-afbc-31535e208a6b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.146967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.152624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.153295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.153582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.153861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.154613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:58.154924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.156328 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.157152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3052-d2ee-4dd6-8e46-abbae26106f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.157798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3052-d2ee-4dd6-8e46-abbae26106f7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.158082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3052-d2ee-4dd6-8e46-abbae26106f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.158359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3052-d2ee-4dd6-8e46-abbae26106f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.165867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 547/1096] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:58.171696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.229320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3052-d2ee-4dd6-8e46-abbae26106f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.230775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3052-d2ee-4dd6-8e46-abbae26106f7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.253371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.269846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf9597-0797-413d-b555-b4d7ded97670 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.303677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c5088-1d01-4210-9f77-3959d1023523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.305069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c5088-1d01-4210-9f77-3959d1023523 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.305347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c5088-1d01-4210-9f77-3959d1023523 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.305623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c5088-1d01-4210-9f77-3959d1023523 None admin] PATH_INFO: `/v3/users/de8da6e3a9c34202abeeb7ec77387805` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.306805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c5088-1d01-4210-9f77-3959d1023523 None admin] RBAC: Authorizing `identity:get_user(user_id=de8da6e3a9c34202abeeb7ec77387805)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.308253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c5088-1d01-4210-9f77-3959d1023523 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.310010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 548/1097] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/users/de8da6e3a9c34202abeeb7ec77387805 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:58.315878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c5088-1d01-4210-9f77-3959d1023523 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.327407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 550/1098] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 11:06:58.332973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3052-d2ee-4dd6-8e46-abbae26106f7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.372810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.412569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c5088-1d01-4210-9f77-3959d1023523 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.428745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3052-d2ee-4dd6-8e46-abbae26106f7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.446542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd83471-031a-4658-912c-f1dc8ae7cb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.447186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd83471-031a-4658-912c-f1dc8ae7cb83 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.447560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd83471-031a-4658-912c-f1dc8ae7cb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.447835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd83471-031a-4658-912c-f1dc8ae7cb83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.450758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.462458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.463235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.463542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.463819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.473480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.474913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:06:58.493742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:58.494681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:58.495808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:06:58.495808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:06:58.495808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:06:58.495808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:58.495808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 551/1099] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:58.500624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.518716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd83471-031a-4658-912c-f1dc8ae7cb83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.520150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd83471-031a-4658-912c-f1dc8ae7cb83 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.577224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.598704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fa470-c9a3-4cfb-b258-ecf770cc9155 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.617189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 549/1100] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:06:58.623146 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd83471-031a-4658-912c-f1dc8ae7cb83 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.631114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.631803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.632095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.632869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.633593 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:58.633903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.635281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.646373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 552/1101] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:58.652467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.686771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.721285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd83471-031a-4658-912c-f1dc8ae7cb83 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.750248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54aa69-3134-4bc2-b956-096c4ae587db None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.754405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb79d84-424d-4631-84f4-62d371e16528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.758178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb79d84-424d-4631-84f4-62d371e16528 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.758178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb79d84-424d-4631-84f4-62d371e16528 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.758178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb79d84-424d-4631-84f4-62d371e16528 None admin] PATH_INFO: `/v3/projects/edc06fb0dec6448386d40635b9e8faba` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.758178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb79d84-424d-4631-84f4-62d371e16528 None admin] RBAC: Authorizing `identity:get_project(project_id=edc06fb0dec6448386d40635b9e8faba)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.758961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb79d84-424d-4631-84f4-62d371e16528 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.760665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 550/1102] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/projects/edc06fb0dec6448386d40635b9e8faba => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:06:58.766631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb79d84-424d-4631-84f4-62d371e16528 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.784350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.785086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.785368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.785646 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.792527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.793951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.794067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:06:58.808383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:06:58.809450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:06:58.810503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 553/1103] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:06:58.815237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:58.864900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb79d84-424d-4631-84f4-62d371e16528 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.897889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.898536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.898819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.899095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.899949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=edc06fb0dec6448386d40635b9e8faba, user.id=de8da6e3a9c34202abeeb7ec77387805) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:58.900260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.901671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.905255 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:58.907913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 551/1104] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/role_assignments?user.id=de8da6e3a9c34202abeeb7ec77387805&scope.project.id=edc06fb0dec6448386d40635b9e8faba&role.id=007af6bd1b734517b934b4cac45faf2c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:58.911641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f34910-31fb-43a3-b163-5e6bcb4fcec2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:58.949358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8470e518-3e11-4277-8f4e-04b8cc62fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:58.950029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470e518-3e11-4277-8f4e-04b8cc62fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:58.950313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470e518-3e11-4277-8f4e-04b8cc62fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:58.950589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8470e518-3e11-4277-8f4e-04b8cc62fdb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:58.951435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470e518-3e11-4277-8f4e-04b8cc62fdb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:58.951751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470e518-3e11-4277-8f4e-04b8cc62fdb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:58.953222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8470e518-3e11-4277-8f4e-04b8cc62fdb5 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:58.966834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 554/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:06:58.973308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:06:59.071752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:06:59.081740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b37fd-9412-47a3-889b-f1fdcbf6f660 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:06:59.117317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337862c-2e8a-4a04-97de-bbd782ab03f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:06:59.118006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337862c-2e8a-4a04-97de-bbd782ab03f2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:06:59.118292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337862c-2e8a-4a04-97de-bbd782ab03f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:06:59.118568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337862c-2e8a-4a04-97de-bbd782ab03f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:06:59.119451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337862c-2e8a-4a04-97de-bbd782ab03f2 None admin] RBAC: Adding query filter params (role.id=131f3f505887436bb2a6499d9ead35c2, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=258ce5f4d6ff400199a80157cc34f5e5) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:06:59.119768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337862c-2e8a-4a04-97de-bbd782ab03f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:06:59.121191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337862c-2e8a-4a04-97de-bbd782ab03f2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:06:59.126949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 552/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:06:58 2022] GET /identity/v3/role_assignments?user.id=258ce5f4d6ff400199a80157cc34f5e5&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=131f3f505887436bb2a6499d9ead35c2 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:00.945253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95eef8-95ef-4345-8dee-115b3d922cb6 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:00.945814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95eef8-95ef-4345-8dee-115b3d922cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:00.945814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95eef8-95ef-4345-8dee-115b3d922cb6 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:00.947195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 555/1107] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:00.952466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83265-074e-4df3-b75d-7df32c784d58 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:00.952899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83265-074e-4df3-b75d-7df32c784d58 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:00.953026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee83265-074e-4df3-b75d-7df32c784d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:00.969335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4ee83265-074e-4df3-b75d-7df32c784d58 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:00.970510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4ee83265-074e-4df3-b75d-7df32c784d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:01.034037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 553/1108] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:01.058079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95eef8-95ef-4345-8dee-115b3d922cb6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:01.156123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95eef8-95ef-4345-8dee-115b3d922cb6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:01.188968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82bc4a-559f-44af-95cd-31918dde9633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:01.189745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82bc4a-559f-44af-95cd-31918dde9633 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:01.190028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82bc4a-559f-44af-95cd-31918dde9633 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:01.190303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82bc4a-559f-44af-95cd-31918dde9633 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:01.191503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82bc4a-559f-44af-95cd-31918dde9633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:01.192928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82bc4a-559f-44af-95cd-31918dde9633 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:01.194568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 556/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:07:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:01.200151 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee83265-074e-4df3-b75d-7df32c784d58 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:01.280939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:01.295532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee83265-074e-4df3-b75d-7df32c784d58 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:01.327356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be454a99-6cb4-4a86-8433-8180d779c444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:01.328037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454a99-6cb4-4a86-8433-8180d779c444 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:01.328325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454a99-6cb4-4a86-8433-8180d779c444 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:01.328603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454a99-6cb4-4a86-8433-8180d779c444 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:01.329303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be454a99-6cb4-4a86-8433-8180d779c444 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:01.330710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be454a99-6cb4-4a86-8433-8180d779c444 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:01.416598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 554/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:07:01 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:07:03.447101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161436ea-6d53-4939-be73-dead324cfd35 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:03.447711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161436ea-6d53-4939-be73-dead324cfd35 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:03.447711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161436ea-6d53-4939-be73-dead324cfd35 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:03.449050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 557/1111] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:03.454055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c2b7d-d38f-4b02-adc3-ac60b77aaaea None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:03.454437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c2b7d-d38f-4b02-adc3-ac60b77aaaea None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:03.454620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c2b7d-d38f-4b02-adc3-ac60b77aaaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:03.471309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a06c2b7d-d38f-4b02-adc3-ac60b77aaaea None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:03.472486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a06c2b7d-d38f-4b02-adc3-ac60b77aaaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:03.535557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 555/1112] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:03.558950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161436ea-6d53-4939-be73-dead324cfd35 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:03.582726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:03.656825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161436ea-6d53-4939-be73-dead324cfd35 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:03.688844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ec75c-ad87-4d68-b2f5-ea419031ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:03.689678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec75c-ad87-4d68-b2f5-ea419031ce1f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:03.689962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec75c-ad87-4d68-b2f5-ea419031ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:03.690236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ec75c-ad87-4d68-b2f5-ea419031ce1f None admin] PATH_INFO: `/v3/roles/9eaabfe45189403f8cee9512fefbba65` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:03.691728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ec75c-ad87-4d68-b2f5-ea419031ce1f None admin] RBAC: Authorizing `identity:get_role(role_id=9eaabfe45189403f8cee9512fefbba65)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:03.693075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61ec75c-ad87-4d68-b2f5-ea419031ce1f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:03.694447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 558/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:03 2022] GET /identity/v3/roles/9eaabfe45189403f8cee9512fefbba65 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:07:03.700140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c2b7d-d38f-4b02-adc3-ac60b77aaaea None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:03.766800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:03.794872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c2b7d-d38f-4b02-adc3-ac60b77aaaea None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:03.829427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaed02-5fb8-45b2-9e6d-00069187fd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:03.830293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaed02-5fb8-45b2-9e6d-00069187fd9d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:03.830579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaed02-5fb8-45b2-9e6d-00069187fd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:03.830855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfaed02-5fb8-45b2-9e6d-00069187fd9d None admin] PATH_INFO: `/v3/users/6d1a2c06e3144ceea42c6339dc458534` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:03.843877 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfaed02-5fb8-45b2-9e6d-00069187fd9d None admin] RBAC: Authorizing `identity:get_user(user_id=6d1a2c06e3144ceea42c6339dc458534)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:03.845305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfaed02-5fb8-45b2-9e6d-00069187fd9d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:03.846930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 556/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:03 2022] GET /identity/v3/users/6d1a2c06e3144ceea42c6339dc458534 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:03.852611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ec75c-ad87-4d68-b2f5-ea419031ce1f None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:03.944912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:03.954780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ec75c-ad87-4d68-b2f5-ea419031ce1f None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:03.986517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990564bb-62d0-47ce-94c6-2f68f2587306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:03.987152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990564bb-62d0-47ce-94c6-2f68f2587306 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:03.987458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990564bb-62d0-47ce-94c6-2f68f2587306 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:03.987731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990564bb-62d0-47ce-94c6-2f68f2587306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:04.051541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990564bb-62d0-47ce-94c6-2f68f2587306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:04.052986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990564bb-62d0-47ce-94c6-2f68f2587306 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:04.136578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:04.146168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 559/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:07:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:07:04.152573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaed02-5fb8-45b2-9e6d-00069187fd9d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:04.250132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfaed02-5fb8-45b2-9e6d-00069187fd9d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:04.277159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:04.282590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1768728-cfed-4d79-88d0-335935a8c9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:04.283466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768728-cfed-4d79-88d0-335935a8c9aa None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:04.283751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768728-cfed-4d79-88d0-335935a8c9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:04.284029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768728-cfed-4d79-88d0-335935a8c9aa None admin] PATH_INFO: `/v3/projects/edc06fb0dec6448386d40635b9e8faba` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:04.285184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1768728-cfed-4d79-88d0-335935a8c9aa None admin] RBAC: Authorizing `identity:get_project(project_id=edc06fb0dec6448386d40635b9e8faba)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:04.286594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1768728-cfed-4d79-88d0-335935a8c9aa None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:04.288307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 557/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:04 2022] GET /identity/v3/projects/edc06fb0dec6448386d40635b9e8faba => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:04.294340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990564bb-62d0-47ce-94c6-2f68f2587306 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:04.388991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990564bb-62d0-47ce-94c6-2f68f2587306 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:04.420847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad8288-c07d-4902-a374-389c1ee52e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:04.421480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad8288-c07d-4902-a374-389c1ee52e47 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:04.421764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad8288-c07d-4902-a374-389c1ee52e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:04.422040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad8288-c07d-4902-a374-389c1ee52e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:04.422860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad8288-c07d-4902-a374-389c1ee52e47 None admin] RBAC: Adding query filter params (role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=edc06fb0dec6448386d40635b9e8faba, user.id=6d1a2c06e3144ceea42c6339dc458534) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:04.423457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad8288-c07d-4902-a374-389c1ee52e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:04.424868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad8288-c07d-4902-a374-389c1ee52e47 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:04.430605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 560/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:07:04 2022] GET /identity/v3/role_assignments?user.id=6d1a2c06e3144ceea42c6339dc458534&scope.project.id=edc06fb0dec6448386d40635b9e8faba&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:06.395856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be53c6e-bbaf-4224-9625-c3bfe9814569 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:06.396404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be53c6e-bbaf-4224-9625-c3bfe9814569 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:06.396404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be53c6e-bbaf-4224-9625-c3bfe9814569 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:06.398272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 558/1118] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:06.403354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037bea5-6d90-49cd-aa3c-e32d8259b381 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:06.403743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037bea5-6d90-49cd-aa3c-e32d8259b381 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:06.403902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037bea5-6d90-49cd-aa3c-e32d8259b381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:06.420011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9037bea5-6d90-49cd-aa3c-e32d8259b381 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:06.421179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9037bea5-6d90-49cd-aa3c-e32d8259b381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:06.468379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:06.484426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 561/1119] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:06.506456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be53c6e-bbaf-4224-9625-c3bfe9814569 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:06.621296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be53c6e-bbaf-4224-9625-c3bfe9814569 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:06.653665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f18235-8ded-4799-bfb5-c3b64dcfd711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:06.654450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f18235-8ded-4799-bfb5-c3b64dcfd711 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:06.654732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f18235-8ded-4799-bfb5-c3b64dcfd711 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:06.655003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f18235-8ded-4799-bfb5-c3b64dcfd711 None admin] PATH_INFO: `/v3/roles/9eaabfe45189403f8cee9512fefbba65` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:06.656478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f18235-8ded-4799-bfb5-c3b64dcfd711 None admin] RBAC: Authorizing `identity:get_role(role_id=9eaabfe45189403f8cee9512fefbba65)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:06.657828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f18235-8ded-4799-bfb5-c3b64dcfd711 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:06.659168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 559/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:06 2022] GET /identity/v3/roles/9eaabfe45189403f8cee9512fefbba65 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:06.664940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037bea5-6d90-49cd-aa3c-e32d8259b381 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:06.759184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037bea5-6d90-49cd-aa3c-e32d8259b381 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:06.791433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8d24-b641-4448-878f-1ed62792f5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:06.792278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8d24-b641-4448-878f-1ed62792f5ac None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:06.792558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8d24-b641-4448-878f-1ed62792f5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:06.792835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8d24-b641-4448-878f-1ed62792f5ac None admin] PATH_INFO: `/v3/users/6d1a2c06e3144ceea42c6339dc458534` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:06.794001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d8d24-b641-4448-878f-1ed62792f5ac None admin] RBAC: Authorizing `identity:get_user(user_id=6d1a2c06e3144ceea42c6339dc458534)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:06.795438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d8d24-b641-4448-878f-1ed62792f5ac None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:06.797012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 562/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:06 2022] GET /identity/v3/users/6d1a2c06e3144ceea42c6339dc458534 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:06.802859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f18235-8ded-4799-bfb5-c3b64dcfd711 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:06.864852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:06.907844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f18235-8ded-4799-bfb5-c3b64dcfd711 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:06.943656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1073222-9e72-4c3a-9841-d7c4ff86990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:06.944559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073222-9e72-4c3a-9841-d7c4ff86990f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:06.944843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073222-9e72-4c3a-9841-d7c4ff86990f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:06.945120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1073222-9e72-4c3a-9841-d7c4ff86990f None admin] PATH_INFO: `/v3/projects/edc06fb0dec6448386d40635b9e8faba` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:06.946294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1073222-9e72-4c3a-9841-d7c4ff86990f None admin] RBAC: Authorizing `identity:get_project(project_id=edc06fb0dec6448386d40635b9e8faba)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:06.947755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1073222-9e72-4c3a-9841-d7c4ff86990f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:06.949464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 560/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:06 2022] GET /identity/v3/projects/edc06fb0dec6448386d40635b9e8faba => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:06.955435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8d24-b641-4448-878f-1ed62792f5ac None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:07.005564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:07.049505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8d24-b641-4448-878f-1ed62792f5ac None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:07.082700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34065936-b842-4a4a-8a30-82bf9253ab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:07.083601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34065936-b842-4a4a-8a30-82bf9253ab5e None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:07.083881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34065936-b842-4a4a-8a30-82bf9253ab5e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:07.084149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34065936-b842-4a4a-8a30-82bf9253ab5e None admin] PATH_INFO: `/v3/projects/edc06fb0dec6448386d40635b9e8faba/users/6d1a2c06e3144ceea42c6339dc458534/roles/9eaabfe45189403f8cee9512fefbba65` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:07.086336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34065936-b842-4a4a-8a30-82bf9253ab5e None admin] RBAC: Authorizing `identity:create_grant(project_id=edc06fb0dec6448386d40635b9e8faba, user_id=6d1a2c06e3144ceea42c6339dc458534, role_id=9eaabfe45189403f8cee9512fefbba65)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:07.087785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34065936-b842-4a4a-8a30-82bf9253ab5e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:07.165346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 563/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:07:06 2022] PUT /identity/v3/projects/edc06fb0dec6448386d40635b9e8faba/users/6d1a2c06e3144ceea42c6339dc458534/roles/9eaabfe45189403f8cee9512fefbba65 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:07:09.158173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48075fbb-b2a5-4af7-8887-7599e9f6d589 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:09.158701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48075fbb-b2a5-4af7-8887-7599e9f6d589 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:09.158795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48075fbb-b2a5-4af7-8887-7599e9f6d589 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:09.160134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 561/1124] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:09.165256 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef737-5bd2-4807-8efd-5c5e5cdb04fa None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:09.165683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef737-5bd2-4807-8efd-5c5e5cdb04fa None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:09.165808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaef737-5bd2-4807-8efd-5c5e5cdb04fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:09.181969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7eaef737-5bd2-4807-8efd-5c5e5cdb04fa None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:09.183131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7eaef737-5bd2-4807-8efd-5c5e5cdb04fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:09.247125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 564/1125] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:09.270500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48075fbb-b2a5-4af7-8887-7599e9f6d589 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:09.368193 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48075fbb-b2a5-4af7-8887-7599e9f6d589 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:09.401285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbcfca-2620-4d5b-a917-a8306f43e3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:09.402127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbcfca-2620-4d5b-a917-a8306f43e3fd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:09.402412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbcfca-2620-4d5b-a917-a8306f43e3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:09.402689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbcfca-2620-4d5b-a917-a8306f43e3fd None admin] PATH_INFO: `/v3/roles/9eaabfe45189403f8cee9512fefbba65` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:09.404187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbcfca-2620-4d5b-a917-a8306f43e3fd None admin] RBAC: Authorizing `identity:get_role(role_id=9eaabfe45189403f8cee9512fefbba65)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:09.405549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedbcfca-2620-4d5b-a917-a8306f43e3fd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:09.406884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 562/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:09 2022] GET /identity/v3/roles/9eaabfe45189403f8cee9512fefbba65 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:09.412436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaef737-5bd2-4807-8efd-5c5e5cdb04fa None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:09.454032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:09.508330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaef737-5bd2-4807-8efd-5c5e5cdb04fa None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:09.539668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f8a9a-5032-429a-9220-9e3c76948a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:09.540509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f8a9a-5032-429a-9220-9e3c76948a59 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:09.540790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f8a9a-5032-429a-9220-9e3c76948a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:09.541061 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f8a9a-5032-429a-9220-9e3c76948a59 None admin] PATH_INFO: `/v3/users/6d1a2c06e3144ceea42c6339dc458534` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:09.542235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f8a9a-5032-429a-9220-9e3c76948a59 None admin] RBAC: Authorizing `identity:get_user(user_id=6d1a2c06e3144ceea42c6339dc458534)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:09.543685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f8a9a-5032-429a-9220-9e3c76948a59 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:09.545287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 565/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:09 2022] GET /identity/v3/users/6d1a2c06e3144ceea42c6339dc458534 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:09.551083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbcfca-2620-4d5b-a917-a8306f43e3fd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:09.554386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:09.646824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbcfca-2620-4d5b-a917-a8306f43e3fd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:09.679400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feac49-d962-4c28-9227-318d69dac9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:09.680028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feac49-d962-4c28-9227-318d69dac9dc None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:09.680317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feac49-d962-4c28-9227-318d69dac9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:09.680602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feac49-d962-4c28-9227-318d69dac9dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:09.744307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63feac49-d962-4c28-9227-318d69dac9dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:09.745733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63feac49-d962-4c28-9227-318d69dac9dc None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:09.747511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:09.840082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 563/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:07:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:07:09.845935 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f8a9a-5032-429a-9220-9e3c76948a59 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:09.944947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f8a9a-5032-429a-9220-9e3c76948a59 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:09.954926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:09.977793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bc141-e663-45b0-98e0-a3601c08481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:09.978908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc141-e663-45b0-98e0-a3601c08481f None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:09.979236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc141-e663-45b0-98e0-a3601c08481f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:09.979707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc141-e663-45b0-98e0-a3601c08481f None admin] PATH_INFO: `/v3/projects/edc06fb0dec6448386d40635b9e8faba` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:09.981226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bc141-e663-45b0-98e0-a3601c08481f None admin] RBAC: Authorizing `identity:get_project(project_id=edc06fb0dec6448386d40635b9e8faba)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:09.982761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bc141-e663-45b0-98e0-a3601c08481f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:09.984581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 566/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:09 2022] GET /identity/v3/projects/edc06fb0dec6448386d40635b9e8faba => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:09.990665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feac49-d962-4c28-9227-318d69dac9dc None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:10.085851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feac49-d962-4c28-9227-318d69dac9dc None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:10.087365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:10.119491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a05f7-0e2b-4cd0-baa4-82fa269d0693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:10.120125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a05f7-0e2b-4cd0-baa4-82fa269d0693 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:10.120484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a05f7-0e2b-4cd0-baa4-82fa269d0693 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:10.121006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a05f7-0e2b-4cd0-baa4-82fa269d0693 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:10.121872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a05f7-0e2b-4cd0-baa4-82fa269d0693 None admin] RBAC: Adding query filter params (role.id=9eaabfe45189403f8cee9512fefbba65, scope.project.id=edc06fb0dec6448386d40635b9e8faba, user.id=6d1a2c06e3144ceea42c6339dc458534) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:10.122358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a05f7-0e2b-4cd0-baa4-82fa269d0693 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:10.124084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a05f7-0e2b-4cd0-baa4-82fa269d0693 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:10.129648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 564/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:07:09 2022] GET /identity/v3/role_assignments?user.id=6d1a2c06e3144ceea42c6339dc458534&scope.project.id=edc06fb0dec6448386d40635b9e8faba&role.id=9eaabfe45189403f8cee9512fefbba65 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:12.126026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9798-c75b-4fee-ac7e-cbfe95c343e7 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:12.126481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9798-c75b-4fee-ac7e-cbfe95c343e7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:12.126600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9798-c75b-4fee-ac7e-cbfe95c343e7 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:12.127956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 567/1131] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:12.133482 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a74ea1-6083-401b-a583-0deff876262f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:12.133886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a74ea1-6083-401b-a583-0deff876262f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:12.134040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a74ea1-6083-401b-a583-0deff876262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:12.150341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-19a74ea1-6083-401b-a583-0deff876262f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:12.151544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-19a74ea1-6083-401b-a583-0deff876262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:12.214991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 565/1132] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:07:12.241991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c9798-c75b-4fee-ac7e-cbfe95c343e7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:12.340357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c9798-c75b-4fee-ac7e-cbfe95c343e7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:12.371363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a9e7b-bfba-4250-94fa-245765638d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:12.372128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a9e7b-bfba-4250-94fa-245765638d35 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:12.372406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a9e7b-bfba-4250-94fa-245765638d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:12.372678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a9e7b-bfba-4250-94fa-245765638d35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:12.373835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a9e7b-bfba-4250-94fa-245765638d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:12.375275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a9e7b-bfba-4250-94fa-245765638d35 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:12.376861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 568/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:07:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:12.383083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a74ea1-6083-401b-a583-0deff876262f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:12.470832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:12.476398 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a74ea1-6083-401b-a583-0deff876262f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:12.508868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210e23-943e-4406-83ca-05d57baafec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:12.509503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210e23-943e-4406-83ca-05d57baafec6 None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:12.509782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210e23-943e-4406-83ca-05d57baafec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:12.510051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210e23-943e-4406-83ca-05d57baafec6 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:12.510754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210e23-943e-4406-83ca-05d57baafec6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:12.512189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210e23-943e-4406-83ca-05d57baafec6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:12.591297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 566/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:07:12 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:07:14.610947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2e1ae-6ce3-4ff0-8204-7d60250e3594 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:14.611520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2e1ae-6ce3-4ff0-8204-7d60250e3594 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:14.611520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2e1ae-6ce3-4ff0-8204-7d60250e3594 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:14.612893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 569/1135] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:14.618089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80b43-329d-40d3-bb9c-a61dcba711eb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:14.618463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80b43-329d-40d3-bb9c-a61dcba711eb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:14.618652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc80b43-329d-40d3-bb9c-a61dcba711eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:14.635333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cdc80b43-329d-40d3-bb9c-a61dcba711eb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:14.636509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cdc80b43-329d-40d3-bb9c-a61dcba711eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:14.699756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 567/1136] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:14.723924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2e1ae-6ce3-4ff0-8204-7d60250e3594 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:14.770001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:14.822415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2e1ae-6ce3-4ff0-8204-7d60250e3594 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:14.853937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0fc0f-29fe-4c5e-a5d6-613d7dd42bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:14.859393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0fc0f-29fe-4c5e-a5d6-613d7dd42bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:14.859393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0fc0f-29fe-4c5e-a5d6-613d7dd42bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:14.859393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0fc0f-29fe-4c5e-a5d6-613d7dd42bf2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:14.859393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0fc0f-29fe-4c5e-a5d6-613d7dd42bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:14.859393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0fc0f-29fe-4c5e-a5d6-613d7dd42bf2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:14.859826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 570/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:07:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:14.865529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc80b43-329d-40d3-bb9c-a61dcba711eb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:14.940139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:14.961095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc80b43-329d-40d3-bb9c-a61dcba711eb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:14.993803 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06b076-cee5-44f3-870c-867f0bebe12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:14.994505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b076-cee5-44f3-870c-867f0bebe12a None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:14.994999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b076-cee5-44f3-870c-867f0bebe12a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:14.995331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b076-cee5-44f3-870c-867f0bebe12a None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:14.996243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06b076-cee5-44f3-870c-867f0bebe12a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:14.998056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06b076-cee5-44f3-870c-867f0bebe12a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:15.092889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 568/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:07:14 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:07:17.131033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfd384-735d-4f08-9a53-2ed8fea443d7 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:17.131615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfd384-735d-4f08-9a53-2ed8fea443d7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:17.131720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfd384-735d-4f08-9a53-2ed8fea443d7 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:17.133026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 571/1139] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:17.138138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522547-6452-4089-8f43-df5c8dc06ddf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:17.138568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522547-6452-4089-8f43-df5c8dc06ddf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:17.138709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522547-6452-4089-8f43-df5c8dc06ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:17.155546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1d522547-6452-4089-8f43-df5c8dc06ddf None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:17.156731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1d522547-6452-4089-8f43-df5c8dc06ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:17.220866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 569/1140] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:17.244428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfd384-735d-4f08-9a53-2ed8fea443d7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:17.342217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfd384-735d-4f08-9a53-2ed8fea443d7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:17.354835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:17.373306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35821711-cc5c-4761-8095-b3d47829b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:17.374080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35821711-cc5c-4761-8095-b3d47829b078 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:17.374359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35821711-cc5c-4761-8095-b3d47829b078 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:17.374634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35821711-cc5c-4761-8095-b3d47829b078 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:17.386466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35821711-cc5c-4761-8095-b3d47829b078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-35821711-cc5c-4761-8095-b3d47829b078 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:07:17.393860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:07:17.394979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:07:17.395940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 572/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:07:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:07:17.401627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d522547-6452-4089-8f43-df5c8dc06ddf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:17.460613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:17.495805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d522547-6452-4089-8f43-df5c8dc06ddf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:17.529347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:17.529995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:17.530279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:17.530553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:17.531318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:17.531627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:17.532975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:17.540670 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 570/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:07:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:17.546176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35821711-cc5c-4761-8095-b3d47829b078 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:17.641092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35821711-cc5c-4761-8095-b3d47829b078 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:17.677449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d182cb1-23cd-486f-aa66-34609ebde2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:17.678817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182cb1-23cd-486f-aa66-34609ebde2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:17.679094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182cb1-23cd-486f-aa66-34609ebde2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:17.679397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d182cb1-23cd-486f-aa66-34609ebde2a9 None admin] PATH_INFO: `/v3/users/e108bedb5c6d470daa99453a88e3f49d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:17.692495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d182cb1-23cd-486f-aa66-34609ebde2a9 None admin] RBAC: Authorizing `identity:get_user(user_id=e108bedb5c6d470daa99453a88e3f49d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:17.694478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d182cb1-23cd-486f-aa66-34609ebde2a9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:17.696136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 573/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:17 2022] GET /identity/v3/users/e108bedb5c6d470daa99453a88e3f49d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:17.701784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:17.796352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c8fef-3e1c-478a-ad16-f4d068abaa46 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:17.821688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:17.829997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e9fc-9ad2-45df-b3a3-08e109098ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:17.830668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e9fc-9ad2-45df-b3a3-08e109098ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:17.830952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e9fc-9ad2-45df-b3a3-08e109098ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:17.831226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e9fc-9ad2-45df-b3a3-08e109098ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:17.895058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077e9fc-9ad2-45df-b3a3-08e109098ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:17.896529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077e9fc-9ad2-45df-b3a3-08e109098ba2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:17.991420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 571/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:07:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:07:17.997195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d182cb1-23cd-486f-aa66-34609ebde2a9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:18.032510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:18.100423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d182cb1-23cd-486f-aa66-34609ebde2a9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:18.133314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9eebf-bbf5-42c8-975f-ebdfd7524c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:18.134159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9eebf-bbf5-42c8-975f-ebdfd7524c8a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:18.134443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9eebf-bbf5-42c8-975f-ebdfd7524c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:18.134721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9eebf-bbf5-42c8-975f-ebdfd7524c8a None admin] PATH_INFO: `/v3/projects/36b13918f01846379a7fe4df25af0c29` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:18.135903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9eebf-bbf5-42c8-975f-ebdfd7524c8a None admin] RBAC: Authorizing `identity:get_project(project_id=36b13918f01846379a7fe4df25af0c29)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:18.137330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9eebf-bbf5-42c8-975f-ebdfd7524c8a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:18.139037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 574/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:17 2022] GET /identity/v3/projects/36b13918f01846379a7fe4df25af0c29 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:18.145282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e9fc-9ad2-45df-b3a3-08e109098ba2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:18.171145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:18.240028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e9fc-9ad2-45df-b3a3-08e109098ba2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:18.272374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87549a21-8b5e-4ef0-9622-4dcaf3f0d4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:18.273014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549a21-8b5e-4ef0-9622-4dcaf3f0d4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:18.273303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549a21-8b5e-4ef0-9622-4dcaf3f0d4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:18.273585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549a21-8b5e-4ef0-9622-4dcaf3f0d4e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:18.274410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87549a21-8b5e-4ef0-9622-4dcaf3f0d4e9 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=36b13918f01846379a7fe4df25af0c29, user.id=e108bedb5c6d470daa99453a88e3f49d) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:18.274728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87549a21-8b5e-4ef0-9622-4dcaf3f0d4e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:18.276170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87549a21-8b5e-4ef0-9622-4dcaf3f0d4e9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:18.288986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 572/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:07:18 2022] GET /identity/v3/role_assignments?user.id=e108bedb5c6d470daa99453a88e3f49d&scope.project.id=36b13918f01846379a7fe4df25af0c29&role.id=007af6bd1b734517b934b4cac45faf2c => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:20.267892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc849-bd82-4cc7-b7c2-6edf6192cdc0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:20.268414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc849-bd82-4cc7-b7c2-6edf6192cdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:20.268414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696bc849-bd82-4cc7-b7c2-6edf6192cdc0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:20.269794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 575/1147] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:20.275604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d20e02-eb39-446e-a68c-96c13387e5ff None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:20.276024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d20e02-eb39-446e-a68c-96c13387e5ff None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:20.276162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d20e02-eb39-446e-a68c-96c13387e5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:20.298758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c6d20e02-eb39-446e-a68c-96c13387e5ff None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:20.299961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c6d20e02-eb39-446e-a68c-96c13387e5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:20.365428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 573/1148] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:20.388147 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bc849-bd82-4cc7-b7c2-6edf6192cdc0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:20.482422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:20.486059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696bc849-bd82-4cc7-b7c2-6edf6192cdc0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:20.518215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7967c8-db96-4f5a-ae10-30999f84d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:20.518995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7967c8-db96-4f5a-ae10-30999f84d99c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:20.519300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7967c8-db96-4f5a-ae10-30999f84d99c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:20.519580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7967c8-db96-4f5a-ae10-30999f84d99c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:20.529980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7967c8-db96-4f5a-ae10-30999f84d99c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-fd7967c8-db96-4f5a-ae10-30999f84d99c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:07:20.537539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:07:20.538667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:07:20.539720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 576/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:07:20.544665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d20e02-eb39-446e-a68c-96c13387e5ff None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:20.564850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:20.639010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d20e02-eb39-446e-a68c-96c13387e5ff None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:20.672190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:20.673101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:20.673380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:20.673651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:20.674388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:20.674693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:20.676076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:20.684334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 574/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:20.690067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7967c8-db96-4f5a-ae10-30999f84d99c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:20.785905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7967c8-db96-4f5a-ae10-30999f84d99c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:20.817977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa6ef3-d912-4620-9a08-dd129e6571de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:20.818863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa6ef3-d912-4620-9a08-dd129e6571de None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:20.819146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa6ef3-d912-4620-9a08-dd129e6571de None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:20.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa6ef3-d912-4620-9a08-dd129e6571de None admin] PATH_INFO: `/v3/users/e108bedb5c6d470daa99453a88e3f49d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:20.820681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa6ef3-d912-4620-9a08-dd129e6571de None admin] RBAC: Authorizing `identity:get_user(user_id=e108bedb5c6d470daa99453a88e3f49d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:20.822093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa6ef3-d912-4620-9a08-dd129e6571de None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:20.823718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 577/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:20 2022] GET /identity/v3/users/e108bedb5c6d470daa99453a88e3f49d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:20.829360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:20.908262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:20.926005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8ea8a-eca7-44ca-a8da-ffac58037326 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:20.957538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e1b16-a22b-494f-89b4-90677151ee07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:20.958397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1b16-a22b-494f-89b4-90677151ee07 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:20.958674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1b16-a22b-494f-89b4-90677151ee07 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:20.958941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1b16-a22b-494f-89b4-90677151ee07 None admin] PATH_INFO: `/v3/projects/36b13918f01846379a7fe4df25af0c29` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:20.960622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e1b16-a22b-494f-89b4-90677151ee07 None admin] RBAC: Authorizing `identity:get_project(project_id=36b13918f01846379a7fe4df25af0c29)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:20.962032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e1b16-a22b-494f-89b4-90677151ee07 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:20.963706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 575/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:20 2022] GET /identity/v3/projects/36b13918f01846379a7fe4df25af0c29 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:20.969497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa6ef3-d912-4620-9a08-dd129e6571de None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:20.998293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:21.067232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa6ef3-d912-4620-9a08-dd129e6571de None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:21.100178 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5116b7-f22a-42b6-8cc8-595497382415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:21.101055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5116b7-f22a-42b6-8cc8-595497382415 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:21.101338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5116b7-f22a-42b6-8cc8-595497382415 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:21.101614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5116b7-f22a-42b6-8cc8-595497382415 None admin] PATH_INFO: `/v3/projects/36b13918f01846379a7fe4df25af0c29/users/e108bedb5c6d470daa99453a88e3f49d/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:21.103566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5116b7-f22a-42b6-8cc8-595497382415 None admin] RBAC: Authorizing `identity:create_grant(project_id=36b13918f01846379a7fe4df25af0c29, user_id=e108bedb5c6d470daa99453a88e3f49d, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:21.104989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5116b7-f22a-42b6-8cc8-595497382415 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:21.176558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 578/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:07:20 2022] PUT /identity/v3/projects/36b13918f01846379a7fe4df25af0c29/users/e108bedb5c6d470daa99453a88e3f49d/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:07:23.200632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe295e5-ab5a-4ae3-b753-7ef7bcc1f472 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:23.201183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe295e5-ab5a-4ae3-b753-7ef7bcc1f472 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:23.201183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe295e5-ab5a-4ae3-b753-7ef7bcc1f472 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:23.202521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 576/1154] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:23.207508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd711-4d51-4e0b-b578-2d86cdb99567 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:23.207885 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd711-4d51-4e0b-b578-2d86cdb99567 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:23.208068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bd711-4d51-4e0b-b578-2d86cdb99567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:23.226009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3c7bd711-4d51-4e0b-b578-2d86cdb99567 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:23.227174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3c7bd711-4d51-4e0b-b578-2d86cdb99567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:23.291508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 579/1155] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:23.314993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe295e5-ab5a-4ae3-b753-7ef7bcc1f472 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:23.412043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe295e5-ab5a-4ae3-b753-7ef7bcc1f472 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:23.443971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99af21-113e-4265-9680-b09993c3ee85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:23.444747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99af21-113e-4265-9680-b09993c3ee85 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:23.445031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99af21-113e-4265-9680-b09993c3ee85 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:23.445312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99af21-113e-4265-9680-b09993c3ee85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:23.455355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99af21-113e-4265-9680-b09993c3ee85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-2e99af21-113e-4265-9680-b09993c3ee85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:07:23.463319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:07:23.464436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:07:23.465450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 577/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:07:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:07:23.470198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bd711-4d51-4e0b-b578-2d86cdb99567 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:23.487695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:23.564927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bd711-4d51-4e0b-b578-2d86cdb99567 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:23.603457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:23.604111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:23.604392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:23.604667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:23.605406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:23.605710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:23.607057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:23.614896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 580/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:07:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:23.620842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99af21-113e-4265-9680-b09993c3ee85 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:23.633316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:23.716493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99af21-113e-4265-9680-b09993c3ee85 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:23.748227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae0078-c653-4923-8fa4-930f329ff706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:23.749076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0078-c653-4923-8fa4-930f329ff706 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:23.749358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0078-c653-4923-8fa4-930f329ff706 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:23.749631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0078-c653-4923-8fa4-930f329ff706 None admin] PATH_INFO: `/v3/users/e108bedb5c6d470daa99453a88e3f49d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:23.750839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae0078-c653-4923-8fa4-930f329ff706 None admin] RBAC: Authorizing `identity:get_user(user_id=e108bedb5c6d470daa99453a88e3f49d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:23.752275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae0078-c653-4923-8fa4-930f329ff706 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:23.753852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 578/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:23 2022] GET /identity/v3/users/e108bedb5c6d470daa99453a88e3f49d => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:23.760165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:23.838330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:23.856329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26359202-d793-490a-9e64-c5bd7ab41d68 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:23.889571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f8f69-7316-477d-8bd1-5c787d7c70cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:23.890202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8f69-7316-477d-8bd1-5c787d7c70cb None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:23.890486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8f69-7316-477d-8bd1-5c787d7c70cb None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:23.890764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8f69-7316-477d-8bd1-5c787d7c70cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:23.955335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f8f69-7316-477d-8bd1-5c787d7c70cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:23.956765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f8f69-7316-477d-8bd1-5c787d7c70cb None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:24.031213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:24.050347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 581/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:07:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:07:24.056838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae0078-c653-4923-8fa4-930f329ff706 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:24.131075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:24.150670 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae0078-c653-4923-8fa4-930f329ff706 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:24.183795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345cec1-15d8-4923-9917-a33d9c184fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:24.184639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cec1-15d8-4923-9917-a33d9c184fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:24.184919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cec1-15d8-4923-9917-a33d9c184fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:24.185194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345cec1-15d8-4923-9917-a33d9c184fe1 None admin] PATH_INFO: `/v3/projects/36b13918f01846379a7fe4df25af0c29` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:24.186341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345cec1-15d8-4923-9917-a33d9c184fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=36b13918f01846379a7fe4df25af0c29)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:24.187787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345cec1-15d8-4923-9917-a33d9c184fe1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:24.189447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 579/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:24 2022] GET /identity/v3/projects/36b13918f01846379a7fe4df25af0c29 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:24.195577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f8f69-7316-477d-8bd1-5c787d7c70cb None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:24.288772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f8f69-7316-477d-8bd1-5c787d7c70cb None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:24.322515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb865a6-efd4-44e0-8f9f-0ea35a5f7cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:24.323148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb865a6-efd4-44e0-8f9f-0ea35a5f7cab None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:24.324194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb865a6-efd4-44e0-8f9f-0ea35a5f7cab None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:24.324194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb865a6-efd4-44e0-8f9f-0ea35a5f7cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:24.324511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb865a6-efd4-44e0-8f9f-0ea35a5f7cab None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=36b13918f01846379a7fe4df25af0c29, user.id=e108bedb5c6d470daa99453a88e3f49d) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:24.324818 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb865a6-efd4-44e0-8f9f-0ea35a5f7cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:24.326230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb865a6-efd4-44e0-8f9f-0ea35a5f7cab None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:24.332318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 582/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:07:24 2022] GET /identity/v3/role_assignments?user.id=e108bedb5c6d470daa99453a88e3f49d&scope.project.id=36b13918f01846379a7fe4df25af0c29&role.id=007af6bd1b734517b934b4cac45faf2c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:26.353409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9860d-663f-4964-afb8-c696a68db0da None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:26.353927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9860d-663f-4964-afb8-c696a68db0da None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:26.353927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9860d-663f-4964-afb8-c696a68db0da None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:26.355219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 580/1162] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:26.360323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afea7f-89fe-4864-8403-b0271ae23530 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:26.360703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afea7f-89fe-4864-8403-b0271ae23530 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:26.360890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afea7f-89fe-4864-8403-b0271ae23530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:26.377129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-35afea7f-89fe-4864-8403-b0271ae23530 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:26.378312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-35afea7f-89fe-4864-8403-b0271ae23530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:26.416711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:26.442794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 583/1163] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:26.465784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9860d-663f-4964-afb8-c696a68db0da None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:26.562582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9860d-663f-4964-afb8-c696a68db0da None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:26.595469 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d28f0-16f3-4b86-8732-e86bc889b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:26.596102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d28f0-16f3-4b86-8732-e86bc889b62c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:26.596391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d28f0-16f3-4b86-8732-e86bc889b62c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:26.596673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d28f0-16f3-4b86-8732-e86bc889b62c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:26.610997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:26.661292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d28f0-16f3-4b86-8732-e86bc889b62c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:26.662759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d28f0-16f3-4b86-8732-e86bc889b62c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:26.757388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 581/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:07:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:07:26.763380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afea7f-89fe-4864-8403-b0271ae23530 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:26.857927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afea7f-89fe-4864-8403-b0271ae23530 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:26.890871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d6888-4b9e-4ea5-92c1-8e521dbfe0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:26.891612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d6888-4b9e-4ea5-92c1-8e521dbfe0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:26.891897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d6888-4b9e-4ea5-92c1-8e521dbfe0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:26.892173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d6888-4b9e-4ea5-92c1-8e521dbfe0c4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:26.898675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d6888-4b9e-4ea5-92c1-8e521dbfe0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:26.900134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d6888-4b9e-4ea5-92c1-8e521dbfe0c4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-9b8d6888-4b9e-4ea5-92c1-8e521dbfe0c4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:07:26.913915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:07:26.915081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:07:26.916006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 584/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:07:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:07:26.920811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d28f0-16f3-4b86-8732-e86bc889b62c None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:26.965343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:27.015693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d28f0-16f3-4b86-8732-e86bc889b62c None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:27.047993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43345aa0-be59-4b65-a9ed-5b07657bcf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:27.048646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43345aa0-be59-4b65-a9ed-5b07657bcf78 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:27.048930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43345aa0-be59-4b65-a9ed-5b07657bcf78 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:27.049206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43345aa0-be59-4b65-a9ed-5b07657bcf78 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:27.049922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43345aa0-be59-4b65-a9ed-5b07657bcf78 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:27.050229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43345aa0-be59-4b65-a9ed-5b07657bcf78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:27.051611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43345aa0-be59-4b65-a9ed-5b07657bcf78 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:27.059234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 582/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Mon Jul 11 11:07:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:29.029723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d4c40-f1e1-44cb-bdeb-41afdefdd0f9 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:29.030313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d4c40-f1e1-44cb-bdeb-41afdefdd0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:29.030313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d4c40-f1e1-44cb-bdeb-41afdefdd0f9 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:29.031662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 585/1167] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:29.037499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e597-a33c-42d7-830d-7bb3aea3dc79 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:29.037923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e597-a33c-42d7-830d-7bb3aea3dc79 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:29.038058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318e597-a33c-42d7-830d-7bb3aea3dc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:29.054520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b318e597-a33c-42d7-830d-7bb3aea3dc79 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:29.055739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b318e597-a33c-42d7-830d-7bb3aea3dc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:29.119440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 583/1168] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:29.142028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d4c40-f1e1-44cb-bdeb-41afdefdd0f9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:29.177239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:29.240118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d4c40-f1e1-44cb-bdeb-41afdefdd0f9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:29.271700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755496a-702c-4572-b903-10403fbb9cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:29.272350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755496a-702c-4572-b903-10403fbb9cce None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:29.272630 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755496a-702c-4572-b903-10403fbb9cce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:29.272898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755496a-702c-4572-b903-10403fbb9cce None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:29.273636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755496a-702c-4572-b903-10403fbb9cce None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:29.274982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755496a-702c-4572-b903-10403fbb9cce None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:29.355572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 586/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Mon Jul 11 11:07:29 2022] POST /identity/v3/domains => generated 267 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:07:31.386123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dce062-9822-4a26-8940-bc0064b74ba8 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:31.386642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dce062-9822-4a26-8940-bc0064b74ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:31.386642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dce062-9822-4a26-8940-bc0064b74ba8 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:31.387982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 584/1170] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:31.393218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68667aa4-77ba-4552-a9a2-7fa86e2f3591 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:31.393636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68667aa4-77ba-4552-a9a2-7fa86e2f3591 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:31.393778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68667aa4-77ba-4552-a9a2-7fa86e2f3591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:31.409976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-68667aa4-77ba-4552-a9a2-7fa86e2f3591 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:31.411154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-68667aa4-77ba-4552-a9a2-7fa86e2f3591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:31.474923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 587/1171] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:31.498335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dce062-9822-4a26-8940-bc0064b74ba8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:31.539944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:31.595235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dce062-9822-4a26-8940-bc0064b74ba8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:31.627748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb04a9d-e2b7-4ca7-b30f-1da1c6eabaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:31.628583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb04a9d-e2b7-4ca7-b30f-1da1c6eabaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:31.628866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb04a9d-e2b7-4ca7-b30f-1da1c6eabaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:31.629141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb04a9d-e2b7-4ca7-b30f-1da1c6eabaf0 None admin] PATH_INFO: `/v3/domains/10f6b646e8624a929bb195275af61b5c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:31.639147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb04a9d-e2b7-4ca7-b30f-1da1c6eabaf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=10f6b646e8624a929bb195275af61b5c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:31.640549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb04a9d-e2b7-4ca7-b30f-1da1c6eabaf0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:31.642133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 585/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Jul 11 11:07:31 2022] GET /identity/v3/domains/10f6b646e8624a929bb195275af61b5c => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:31.647835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68667aa4-77ba-4552-a9a2-7fa86e2f3591 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:31.671655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:31.742723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68667aa4-77ba-4552-a9a2-7fa86e2f3591 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:31.774775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee6131-95fe-4a9a-9ec8-d25baa1d486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:31.775445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6131-95fe-4a9a-9ec8-d25baa1d486f None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:31.775730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6131-95fe-4a9a-9ec8-d25baa1d486f None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:31.776005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6131-95fe-4a9a-9ec8-d25baa1d486f None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:31.776703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee6131-95fe-4a9a-9ec8-d25baa1d486f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:31.778111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee6131-95fe-4a9a-9ec8-d25baa1d486f None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:31.875377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 588/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:07:31 2022] POST /identity/v3/projects => generated 361 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:07:33.898417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d6948-18cd-4102-9d35-5660c4212524 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:33.898992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d6948-18cd-4102-9d35-5660c4212524 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:33.898992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d6948-18cd-4102-9d35-5660c4212524 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:33.900343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 586/1174] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:33.905558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19887c63-0ced-414d-995b-16dd0ba8d347 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:33.905982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19887c63-0ced-414d-995b-16dd0ba8d347 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:33.906117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19887c63-0ced-414d-995b-16dd0ba8d347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:33.923830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-19887c63-0ced-414d-995b-16dd0ba8d347 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:33.925109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-19887c63-0ced-414d-995b-16dd0ba8d347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:33.990508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 589/1175] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:34.015010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d6948-18cd-4102-9d35-5660c4212524 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:34.109779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:34.113514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d6948-18cd-4102-9d35-5660c4212524 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:34.146100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b04f97-973d-4fba-89dc-4b1ebdd958be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:34.146953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b04f97-973d-4fba-89dc-4b1ebdd958be None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:34.147238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b04f97-973d-4fba-89dc-4b1ebdd958be None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:34.147541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b04f97-973d-4fba-89dc-4b1ebdd958be None admin] PATH_INFO: `/v3/domains/10f6b646e8624a929bb195275af61b5c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:34.148754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b04f97-973d-4fba-89dc-4b1ebdd958be None admin] RBAC: Authorizing `identity:get_domain(domain_id=10f6b646e8624a929bb195275af61b5c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:34.150120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b04f97-973d-4fba-89dc-4b1ebdd958be None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:34.151747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 587/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Mon Jul 11 11:07:34 2022] GET /identity/v3/domains/10f6b646e8624a929bb195275af61b5c => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:34.157452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19887c63-0ced-414d-995b-16dd0ba8d347 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:34.165883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:34.252238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19887c63-0ced-414d-995b-16dd0ba8d347 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:34.285064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cc7f8-314f-4148-a0ad-5a73e7c2b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:34.285715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cc7f8-314f-4148-a0ad-5a73e7c2b251 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:34.285998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cc7f8-314f-4148-a0ad-5a73e7c2b251 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:34.286274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cc7f8-314f-4148-a0ad-5a73e7c2b251 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:34.287613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cc7f8-314f-4148-a0ad-5a73e7c2b251 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:34.289025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cc7f8-314f-4148-a0ad-5a73e7c2b251 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:34.382337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 590/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:07:34 2022] POST /identity/v3/users => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:07:36.417841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c74bf-3189-4b8c-a94d-9d9a3e8543a4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:36.418359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c74bf-3189-4b8c-a94d-9d9a3e8543a4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:36.418472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c74bf-3189-4b8c-a94d-9d9a3e8543a4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:36.419775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 588/1178] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:36.425218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d0f7-50d7-4323-b5aa-a3a1f50533c8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:36.425618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d0f7-50d7-4323-b5aa-a3a1f50533c8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:36.425777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d0f7-50d7-4323-b5aa-a3a1f50533c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:36.442586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5bc2d0f7-50d7-4323-b5aa-a3a1f50533c8 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:36.443798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5bc2d0f7-50d7-4323-b5aa-a3a1f50533c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:36.497961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:36.507921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 591/1179] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:36.531431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c74bf-3189-4b8c-a94d-9d9a3e8543a4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:36.627435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c74bf-3189-4b8c-a94d-9d9a3e8543a4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:36.660606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735106-c563-46f2-bc70-daeff30fc32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:36.661455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735106-c563-46f2-bc70-daeff30fc32a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:36.662471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735106-c563-46f2-bc70-daeff30fc32a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:36.662744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735106-c563-46f2-bc70-daeff30fc32a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:36.675917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735106-c563-46f2-bc70-daeff30fc32a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-46735106-c563-46f2-bc70-daeff30fc32a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:07:36.683912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:07:36.685101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:07:36.686402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 589/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:07:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:07:36.691104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d0f7-50d7-4323-b5aa-a3a1f50533c8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:36.785526 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d0f7-50d7-4323-b5aa-a3a1f50533c8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:36.819006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:36.819684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:36.819967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:36.820240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:36.820986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:36.821293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:36.822743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:36.830522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 592/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:36.836896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735106-c563-46f2-bc70-daeff30fc32a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:36.864229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:36.931683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735106-c563-46f2-bc70-daeff30fc32a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:36.963801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23f2a-23fb-4ec5-826b-79673682d162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:36.964972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23f2a-23fb-4ec5-826b-79673682d162 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:36.965254 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23f2a-23fb-4ec5-826b-79673682d162 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:36.965528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d23f2a-23fb-4ec5-826b-79673682d162 None admin] PATH_INFO: `/v3/users/67656129fa684d5bacf901223a1e50c1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:36.977962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d23f2a-23fb-4ec5-826b-79673682d162 None admin] RBAC: Authorizing `identity:get_user(user_id=67656129fa684d5bacf901223a1e50c1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:36.979409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d23f2a-23fb-4ec5-826b-79673682d162 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:36.981003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 590/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:36 2022] GET /identity/v3/users/67656129fa684d5bacf901223a1e50c1 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:36.986853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:37.042335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:37.086510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ead2e-8d29-451f-addf-b8fe7100eaf1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:37.118168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8779b7-62c7-4171-a646-ea77a5c5008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:37.118803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8779b7-62c7-4171-a646-ea77a5c5008d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:37.119087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8779b7-62c7-4171-a646-ea77a5c5008d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:37.119394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8779b7-62c7-4171-a646-ea77a5c5008d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:37.184560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8779b7-62c7-4171-a646-ea77a5c5008d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:37.185932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8779b7-62c7-4171-a646-ea77a5c5008d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:37.236382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:37.279642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 593/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:07:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:07:37.285712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23f2a-23fb-4ec5-826b-79673682d162 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:37.370126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:37.384348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d23f2a-23fb-4ec5-826b-79673682d162 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:37.417625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3073e-d7a5-4e20-9a7e-9532d352713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:37.418471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3073e-d7a5-4e20-9a7e-9532d352713a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:37.419020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3073e-d7a5-4e20-9a7e-9532d352713a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:37.419020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3073e-d7a5-4e20-9a7e-9532d352713a None admin] PATH_INFO: `/v3/projects/290cb4d0864a4dfbb890f5b29646ef06` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:37.420188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3073e-d7a5-4e20-9a7e-9532d352713a None admin] RBAC: Authorizing `identity:get_project(project_id=290cb4d0864a4dfbb890f5b29646ef06)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:37.421599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3073e-d7a5-4e20-9a7e-9532d352713a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:37.423551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 591/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:37 2022] GET /identity/v3/projects/290cb4d0864a4dfbb890f5b29646ef06 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:37.429610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8779b7-62c7-4171-a646-ea77a5c5008d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:37.523630 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8779b7-62c7-4171-a646-ea77a5c5008d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:37.557273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa253f7f-b6bd-4bc1-8231-bdf29bbf39ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:37.557910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa253f7f-b6bd-4bc1-8231-bdf29bbf39ec None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:37.558194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa253f7f-b6bd-4bc1-8231-bdf29bbf39ec None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:37.558470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa253f7f-b6bd-4bc1-8231-bdf29bbf39ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:37.559321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa253f7f-b6bd-4bc1-8231-bdf29bbf39ec None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=290cb4d0864a4dfbb890f5b29646ef06, user.id=67656129fa684d5bacf901223a1e50c1) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:37.559752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa253f7f-b6bd-4bc1-8231-bdf29bbf39ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:37.561163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa253f7f-b6bd-4bc1-8231-bdf29bbf39ec None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:37.566452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 594/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:07:37 2022] GET /identity/v3/role_assignments?user.id=67656129fa684d5bacf901223a1e50c1&scope.project.id=290cb4d0864a4dfbb890f5b29646ef06&role.id=007af6bd1b734517b934b4cac45faf2c => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:39.636268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62eb2-34b9-464e-8729-92987a65d616 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:39.636836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62eb2-34b9-464e-8729-92987a65d616 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:39.636836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62eb2-34b9-464e-8729-92987a65d616 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:39.638170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 592/1186] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:39.643541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5624dc0-9b4d-4c8a-b0a8-52d1145bc59a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:39.643988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5624dc0-9b4d-4c8a-b0a8-52d1145bc59a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:39.644090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5624dc0-9b4d-4c8a-b0a8-52d1145bc59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:39.662575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b5624dc0-9b4d-4c8a-b0a8-52d1145bc59a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:39.663793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b5624dc0-9b4d-4c8a-b0a8-52d1145bc59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:39.676868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:39.728389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 595/1187] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:39.750125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb62eb2-34b9-464e-8729-92987a65d616 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:39.846466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb62eb2-34b9-464e-8729-92987a65d616 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:39.877837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47126bee-c873-4793-8119-8c0b78eb8434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:39.878611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47126bee-c873-4793-8119-8c0b78eb8434 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:39.878891 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47126bee-c873-4793-8119-8c0b78eb8434 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:39.879159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47126bee-c873-4793-8119-8c0b78eb8434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:39.889378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47126bee-c873-4793-8119-8c0b78eb8434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-47126bee-c873-4793-8119-8c0b78eb8434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:07:39.897364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:07:39.898565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:07:39.899710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 593/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:07:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:07:39.904395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5624dc0-9b4d-4c8a-b0a8-52d1145bc59a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:39.997372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5624dc0-9b4d-4c8a-b0a8-52d1145bc59a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:40.029803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:40.030442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:40.030721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:40.030992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:40.031756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:40.032060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:40.033395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:40.041065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 596/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:40.046666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47126bee-c873-4793-8119-8c0b78eb8434 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:40.062598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:40.144318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47126bee-c873-4793-8119-8c0b78eb8434 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:40.176486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c03da-5963-43bd-8b3f-21a9db8ffe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:40.177861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c03da-5963-43bd-8b3f-21a9db8ffe9d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:40.178144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c03da-5963-43bd-8b3f-21a9db8ffe9d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:40.178420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c03da-5963-43bd-8b3f-21a9db8ffe9d None admin] PATH_INFO: `/v3/users/67656129fa684d5bacf901223a1e50c1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:40.179622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c03da-5963-43bd-8b3f-21a9db8ffe9d None admin] RBAC: Authorizing `identity:get_user(user_id=67656129fa684d5bacf901223a1e50c1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:40.181041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c03da-5963-43bd-8b3f-21a9db8ffe9d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:40.182647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 594/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:40 2022] GET /identity/v3/users/67656129fa684d5bacf901223a1e50c1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:40.188494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:40.201322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:40.284052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d5e5a-dfa0-48f7-8954-9bd632ac0065 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:40.315919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74a7b1-b9ee-4eef-978d-351c795068b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:40.316753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a7b1-b9ee-4eef-978d-351c795068b7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:40.317028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a7b1-b9ee-4eef-978d-351c795068b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:40.317298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a7b1-b9ee-4eef-978d-351c795068b7 None admin] PATH_INFO: `/v3/projects/290cb4d0864a4dfbb890f5b29646ef06` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:40.319036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74a7b1-b9ee-4eef-978d-351c795068b7 None admin] RBAC: Authorizing `identity:get_project(project_id=290cb4d0864a4dfbb890f5b29646ef06)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:40.320483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74a7b1-b9ee-4eef-978d-351c795068b7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:40.322139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 597/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:40 2022] GET /identity/v3/projects/290cb4d0864a4dfbb890f5b29646ef06 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:40.327940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c03da-5963-43bd-8b3f-21a9db8ffe9d None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:40.406415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:40.423819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c03da-5963-43bd-8b3f-21a9db8ffe9d None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:40.461062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3bec0-0fd6-4bd6-bf70-e2f8834b266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:40.461951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bec0-0fd6-4bd6-bf70-e2f8834b266b None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:40.462235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bec0-0fd6-4bd6-bf70-e2f8834b266b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:40.462509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bec0-0fd6-4bd6-bf70-e2f8834b266b None admin] PATH_INFO: `/v3/projects/290cb4d0864a4dfbb890f5b29646ef06/users/67656129fa684d5bacf901223a1e50c1/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:40.464462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3bec0-0fd6-4bd6-bf70-e2f8834b266b None admin] RBAC: Authorizing `identity:create_grant(project_id=290cb4d0864a4dfbb890f5b29646ef06, user_id=67656129fa684d5bacf901223a1e50c1, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:40.465887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3bec0-0fd6-4bd6-bf70-e2f8834b266b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:40.518900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 595/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:07:40 2022] PUT /identity/v3/projects/290cb4d0864a4dfbb890f5b29646ef06/users/67656129fa684d5bacf901223a1e50c1/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:07:42.507109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2b28-5b91-4be8-b687-e461d61f1663 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:42.507680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2b28-5b91-4be8-b687-e461d61f1663 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:42.507680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2b28-5b91-4be8-b687-e461d61f1663 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:42.509045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 598/1193] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:07:42.514080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef7628-41b6-4ffc-9f18-9ee0f79e469d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:42.514471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef7628-41b6-4ffc-9f18-9ee0f79e469d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:42.514647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef7628-41b6-4ffc-9f18-9ee0f79e469d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:42.532156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-53ef7628-41b6-4ffc-9f18-9ee0f79e469d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:07:42.533352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-53ef7628-41b6-4ffc-9f18-9ee0f79e469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:07:42.596824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 596/1194] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:07:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:07:42.620563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d2b28-5b91-4be8-b687-e461d61f1663 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:42.697292 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:42.718849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d2b28-5b91-4be8-b687-e461d61f1663 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:42.751320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c43e8e-6105-471b-a50c-4a83f0339753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:42.752094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c43e8e-6105-471b-a50c-4a83f0339753 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:42.752381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c43e8e-6105-471b-a50c-4a83f0339753 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:42.752660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c43e8e-6105-471b-a50c-4a83f0339753 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:42.763168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c43e8e-6105-471b-a50c-4a83f0339753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-b5c43e8e-6105-471b-a50c-4a83f0339753 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:07:42.771082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:07:42.772216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:07:42.773106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 599/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:07:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:07:42.778406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef7628-41b6-4ffc-9f18-9ee0f79e469d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:42.844648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:42.873746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef7628-41b6-4ffc-9f18-9ee0f79e469d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:42.905905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:42.906560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:42.906844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:42.907121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:42.907889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:42.908200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:42.909579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:42.917802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 597/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:42.923857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c43e8e-6105-471b-a50c-4a83f0339753 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:43.019106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c43e8e-6105-471b-a50c-4a83f0339753 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:43.050841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a1791-5b2d-4b20-a85c-2ce68a2cddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:43.051783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1791-5b2d-4b20-a85c-2ce68a2cddab None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:43.052559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1791-5b2d-4b20-a85c-2ce68a2cddab None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:43.052826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a1791-5b2d-4b20-a85c-2ce68a2cddab None admin] PATH_INFO: `/v3/users/67656129fa684d5bacf901223a1e50c1` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:43.054020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a1791-5b2d-4b20-a85c-2ce68a2cddab None admin] RBAC: Authorizing `identity:get_user(user_id=67656129fa684d5bacf901223a1e50c1)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:43.055454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a1791-5b2d-4b20-a85c-2ce68a2cddab None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:43.057058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 600/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Mon Jul 11 11:07:42 2022] GET /identity/v3/users/67656129fa684d5bacf901223a1e50c1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:43.062638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:43.157767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6432389-9524-4a2b-93cc-b6b16ee62e95 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:43.190523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998283f-e8ad-4dea-930e-0a1f8b149d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:43.191158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998283f-e8ad-4dea-930e-0a1f8b149d67 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:43.191472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998283f-e8ad-4dea-930e-0a1f8b149d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:43.191752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998283f-e8ad-4dea-930e-0a1f8b149d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:43.194308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:43.256100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998283f-e8ad-4dea-930e-0a1f8b149d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:43.257527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998283f-e8ad-4dea-930e-0a1f8b149d67 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:43.349804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 598/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:07:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:07:43.355456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a1791-5b2d-4b20-a85c-2ce68a2cddab None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:43.364812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:43.451169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a1791-5b2d-4b20-a85c-2ce68a2cddab None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:43.482407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef242a-0c78-46ab-bcb1-d32a37d9e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:43.483271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef242a-0c78-46ab-bcb1-d32a37d9e283 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:43.483556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef242a-0c78-46ab-bcb1-d32a37d9e283 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:43.483825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef242a-0c78-46ab-bcb1-d32a37d9e283 None admin] PATH_INFO: `/v3/projects/290cb4d0864a4dfbb890f5b29646ef06` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:43.484985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef242a-0c78-46ab-bcb1-d32a37d9e283 None admin] RBAC: Authorizing `identity:get_project(project_id=290cb4d0864a4dfbb890f5b29646ef06)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:43.486396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef242a-0c78-46ab-bcb1-d32a37d9e283 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:43.488181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 601/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Mon Jul 11 11:07:43 2022] GET /identity/v3/projects/290cb4d0864a4dfbb890f5b29646ef06 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:43.494906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998283f-e8ad-4dea-930e-0a1f8b149d67 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:07:43.526996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:43.591874 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998283f-e8ad-4dea-930e-0a1f8b149d67 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:07:43.623589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525076-7f40-4642-b6d8-8ee61f85f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:07:43.624174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525076-7f40-4642-b6d8-8ee61f85f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:07:43.624460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525076-7f40-4642-b6d8-8ee61f85f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:07:43.624735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525076-7f40-4642-b6d8-8ee61f85f6f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:07:43.625553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525076-7f40-4642-b6d8-8ee61f85f6f2 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=290cb4d0864a4dfbb890f5b29646ef06, user.id=67656129fa684d5bacf901223a1e50c1) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:07:43.625866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525076-7f40-4642-b6d8-8ee61f85f6f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:07:43.627288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525076-7f40-4642-b6d8-8ee61f85f6f2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:07:43.633437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 599/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:07:43 2022] GET /identity/v3/role_assignments?user.id=67656129fa684d5bacf901223a1e50c1&scope.project.id=290cb4d0864a4dfbb890f5b29646ef06&role.id=007af6bd1b734517b934b4cac45faf2c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:07:53.478399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=63 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:07:53.631092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=51 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:02.264899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a79341-1419-412e-a3dc-6bb54331e061 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:02.265490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a79341-1419-412e-a3dc-6bb54331e061 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:02.265490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a79341-1419-412e-a3dc-6bb54331e061 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:02.266813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 602/1201] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:02.271917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e018-a8c7-407e-bc6d-490822cee54f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:02.272350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e018-a8c7-407e-bc6d-490822cee54f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:02.272483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38e018-a8c7-407e-bc6d-490822cee54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:02.289448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1d38e018-a8c7-407e-bc6d-490822cee54f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:02.294122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1d38e018-a8c7-407e-bc6d-490822cee54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:02.360079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 600/1202] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:02.384237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a79341-1419-412e-a3dc-6bb54331e061 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:02.483895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a79341-1419-412e-a3dc-6bb54331e061 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:02.516409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:02.517925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:02.517925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:02.517925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:02.527896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:02.529319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:02.546434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:02.549828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:02.551065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:02.551065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:02.551065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:02.551065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:02.551065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 603/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:02.554306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38e018-a8c7-407e-bc6d-490822cee54f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:02.649763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38e018-a8c7-407e-bc6d-490822cee54f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:02.677527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:02.683022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af149b0-e4ad-442f-9256-c2e03c53d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:02.683684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af149b0-e4ad-442f-9256-c2e03c53d725 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:02.683963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af149b0-e4ad-442f-9256-c2e03c53d725 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:02.684234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af149b0-e4ad-442f-9256-c2e03c53d725 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:02.684942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af149b0-e4ad-442f-9256-c2e03c53d725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:02.685252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af149b0-e4ad-442f-9256-c2e03c53d725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:02.686661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af149b0-e4ad-442f-9256-c2e03c53d725 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:02.697584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 601/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:02.703205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:02.754296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:02.800400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af2716-7594-478f-8547-f83e51d704c8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:02.836619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eceb50-1f2c-476e-8a10-378d55879ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:02.840766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eceb50-1f2c-476e-8a10-378d55879ddc None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:02.840766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eceb50-1f2c-476e-8a10-378d55879ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:02.840766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eceb50-1f2c-476e-8a10-378d55879ddc None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:02.840766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eceb50-1f2c-476e-8a10-378d55879ddc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:02.840766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eceb50-1f2c-476e-8a10-378d55879ddc None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:02.987293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 604/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Mon Jul 11 11:09:02 2022] POST /identity/v3/users => generated 251 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:09:05.013121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19a8e-8016-455b-8215-98fb242b883d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:05.013791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19a8e-8016-455b-8215-98fb242b883d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:05.013791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19a8e-8016-455b-8215-98fb242b883d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:05.015028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 602/1206] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:05.020232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7df7c1-bf5d-42ee-9ca2-ece7d1cbbf7a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:05.023363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7df7c1-bf5d-42ee-9ca2-ece7d1cbbf7a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:05.023363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7df7c1-bf5d-42ee-9ca2-ece7d1cbbf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:05.037377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cf7df7c1-bf5d-42ee-9ca2-ece7d1cbbf7a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:05.038587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cf7df7c1-bf5d-42ee-9ca2-ece7d1cbbf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:05.105251 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 605/1207] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:09:05.129077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19a8e-8016-455b-8215-98fb242b883d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:05.228346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19a8e-8016-455b-8215-98fb242b883d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:05.260325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eace0ce-6bc8-465f-8701-98257da2ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:05.264609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eace0ce-6bc8-465f-8701-98257da2ea90 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:05.264609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eace0ce-6bc8-465f-8701-98257da2ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:05.264609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eace0ce-6bc8-465f-8701-98257da2ea90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:05.271942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eace0ce-6bc8-465f-8701-98257da2ea90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3eace0ce-6bc8-465f-8701-98257da2ea90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:09:05.279604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:05.280714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:05.281837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 603/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:09:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:05.287510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7df7c1-bf5d-42ee-9ca2-ece7d1cbbf7a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:05.325801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:05.384048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7df7c1-bf5d-42ee-9ca2-ece7d1cbbf7a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:05.417387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:05.418085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:05.418458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:05.418738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:05.419554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:05.419866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:05.421476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:05.429872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 606/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:09:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:05.435767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eace0ce-6bc8-465f-8701-98257da2ea90 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:05.484470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:05.531396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eace0ce-6bc8-465f-8701-98257da2ea90 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:05.564362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:05.567104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:05.567104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:05.567104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:05.575349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:05.576780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:05.594271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:05.599131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:05.600698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:05.600698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:05.600698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:05.600698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:05.600698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 604/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:05.602391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:05.698004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:05.700963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76db564-6492-45b3-9164-bae61e0469d4 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:05.741485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:05.742186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:05.742475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:05.742757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:05.743509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:05.743825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:05.745189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:05.757011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 607/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:05.762875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:05.859425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc066a5c-6259-45b8-b3be-e62b5c3fd201 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:05.860868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:05.891779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:05.892567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:05.892853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:05.893128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:05.900340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:05.901765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:09:05.916599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:05.917751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:05.918959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 605/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:05.923994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:06.025748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299f544-1ee6-48d2-9330-f416696f4ab1 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:06.060801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:06.061466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:06.061749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:06.062023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:06.062995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:06.063293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:06.064708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:06.082374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 608/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 11 11:09:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:06.088755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:06.185896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb939526-1e12-4243-92ab-8cb8d791d577 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:06.219102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa555254-8b4d-4c22-8158-9df49a5d0e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:06.219771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa555254-8b4d-4c22-8158-9df49a5d0e4f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:06.220057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa555254-8b4d-4c22-8158-9df49a5d0e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:06.220335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa555254-8b4d-4c22-8158-9df49a5d0e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:06.248403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:06.285162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa555254-8b4d-4c22-8158-9df49a5d0e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:06.286599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa555254-8b4d-4c22-8158-9df49a5d0e4f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:06.382979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 606/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:09:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:09:06.389020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:06.407510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:06.484409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b847f-0b58-41da-b194-091ad768f840 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:06.518008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:06.518796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:06.519080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:06.519392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:06.529442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:06.530875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:06.548219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:06.555353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:06.556479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:06.556479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:06.556479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:06.556479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:06.556479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 609/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:06.556842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa555254-8b4d-4c22-8158-9df49a5d0e4f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:06.614931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:06.652116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa555254-8b4d-4c22-8158-9df49a5d0e4f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:06.684638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:06.685300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:06.685580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:06.685856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:06.686576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:06.686884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:06.688273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:06.699770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 607/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:09:06.705411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:06.773081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:06.803673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b387ab-49ee-4ddc-9620-5425a8515a8a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:06.839764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:06.840553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:06.840840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:06.841116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:06.847889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:06.850006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:09:06.864447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:06.865465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:06.866696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 610/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:06.871591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:06.968312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7eaf95-2b7e-4d3d-99e3-2219dd68ae94 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:06.990614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:07.001861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47160e9-6e0c-4a9f-b431-1a477e309190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:07.002524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47160e9-6e0c-4a9f-b431-1a477e309190 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:07.002834 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47160e9-6e0c-4a9f-b431-1a477e309190 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:07.003109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47160e9-6e0c-4a9f-b431-1a477e309190 None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:07.003928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47160e9-6e0c-4a9f-b431-1a477e309190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:07.004268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47160e9-6e0c-4a9f-b431-1a477e309190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:07.005725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47160e9-6e0c-4a9f-b431-1a477e309190 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:07.017509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 608/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:07.023804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:07.124462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f1e58-ef75-4019-b71f-8c7cc1956fb8 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:07.151763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:07.157368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bc8b5-6dd6-4471-9457-1d644c7228da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:07.158013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc8b5-6dd6-4471-9457-1d644c7228da None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:07.158294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc8b5-6dd6-4471-9457-1d644c7228da None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:07.158568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bc8b5-6dd6-4471-9457-1d644c7228da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:07.159420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bc8b5-6dd6-4471-9457-1d644c7228da None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=94f2124d6d4e46ca9baa9728741b0e4c) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:07.159735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bc8b5-6dd6-4471-9457-1d644c7228da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:07.161708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bc8b5-6dd6-4471-9457-1d644c7228da None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:07.166960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 611/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:09:07 2022] GET /identity/v3/role_assignments?user.id=94f2124d6d4e46ca9baa9728741b0e4c&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:09.153665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1254ab-6a96-48be-8a13-712c124cf1ce None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:09.154228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1254ab-6a96-48be-8a13-712c124cf1ce None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:09.154228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1254ab-6a96-48be-8a13-712c124cf1ce None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:09.155604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 609/1220] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:09.161318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18dd4f-8212-4b05-9aba-6447aadb24a2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:09.168022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18dd4f-8212-4b05-9aba-6447aadb24a2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:09.168022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18dd4f-8212-4b05-9aba-6447aadb24a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:09.179426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1a18dd4f-8212-4b05-9aba-6447aadb24a2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:09.180596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1a18dd4f-8212-4b05-9aba-6447aadb24a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:09.244635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 612/1221] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 11:09:09.266766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1254ab-6a96-48be-8a13-712c124cf1ce None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:09.367799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1254ab-6a96-48be-8a13-712c124cf1ce None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:09.400662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f35fc-f754-43a3-819f-3c0c69101cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:09.403019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f35fc-f754-43a3-819f-3c0c69101cec None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:09.403019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f35fc-f754-43a3-819f-3c0c69101cec None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:09.403019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f35fc-f754-43a3-819f-3c0c69101cec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:09.412403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f35fc-f754-43a3-819f-3c0c69101cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-3a1f35fc-f754-43a3-819f-3c0c69101cec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:09:09.420064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:09.421407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:09.422396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 610/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:09:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:09.427647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18dd4f-8212-4b05-9aba-6447aadb24a2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:09.523163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18dd4f-8212-4b05-9aba-6447aadb24a2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:09.546413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:09.556434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:09.557146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:09.557432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:09.557707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:09.558454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:09.558760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:09.560145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:09.568459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 613/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:09:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:09.574184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f35fc-f754-43a3-819f-3c0c69101cec None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:09.618450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:09.673292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f35fc-f754-43a3-819f-3c0c69101cec None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:09.708909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:09.709693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:09.709970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:09.713886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:09.720524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:09.721946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:09.739274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:09.740361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:09.741658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:09.741658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:09.741658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:09.741658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:09.741658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 611/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:09.746354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:09.844987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f3a82-e566-4e34-bbd4-47c96eabbf52 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:09.877961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:09.878650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:09.878933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:09.879228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:09.879971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:09.880279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:09.881661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:09.893068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 614/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:09.898960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:09.990734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:09.997297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426c015-2fe4-4712-81da-504a6097c833 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:10.030692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:10.031522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:10.031805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:10.032080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:10.039843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:10.041268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:09:10.056005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:10.062752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:10.063991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 612/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:10.064734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:10.110012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:10.160978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152200a2-38c7-4836-bacd-e273dcb30e37 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:10.192808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:10.193467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:10.193749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:10.194025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:10.194855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:10.195164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:10.196610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:10.213472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 615/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 11 11:09:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:10.219370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:10.318238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc613-2550-4a36-8b11-a74e249f0ccd None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:10.350932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:10.351742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:10.352023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:10.354156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:10.362448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:10.363943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:10.380379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:10.381473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:10.382629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:10.382629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:10.382629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:10.382629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:10.382629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 613/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:10.387199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:10.484819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55e473-de33-43a1-9098-92c5038b41a2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:10.487045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:10.521920 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:10.522581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:10.522867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:10.523148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:10.523902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:10.524216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:10.525673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:10.537659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 616/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:10.543345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:10.552854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:10.646533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c6d02-ff57-4db7-a69e-f1e44f6eb304 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:10.686942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:10.687772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:10.688045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:10.688318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:10.696483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:10.697970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:09:10.712620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:10.713689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:10.714815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 614/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:09:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:10.721409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:10.820691 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466687ab-50f3-443c-838a-46343adbff0c None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:10.854368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf342761-8494-4be0-bf1c-b105dbe70870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:10.855016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342761-8494-4be0-bf1c-b105dbe70870 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:10.855322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342761-8494-4be0-bf1c-b105dbe70870 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:10.855595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342761-8494-4be0-bf1c-b105dbe70870 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:10.856374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342761-8494-4be0-bf1c-b105dbe70870 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:10.856801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342761-8494-4be0-bf1c-b105dbe70870 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:10.858157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342761-8494-4be0-bf1c-b105dbe70870 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:10.874451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 617/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:09:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:10.880497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:10.946530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:10.980789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c797a6-c95a-4c98-895d-543a0dc9610a None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:11.014000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2ef28-eeb5-445b-a68f-e84d4fb5b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:11.014912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2ef28-eeb5-445b-a68f-e84d4fb5b4ad None admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:11.015198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2ef28-eeb5-445b-a68f-e84d4fb5b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:11.015503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2ef28-eeb5-445b-a68f-e84d4fb5b4ad None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/94f2124d6d4e46ca9baa9728741b0e4c/roles/d82425658c8545e6b546d601ef7b9b58` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:11.030126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2ef28-eeb5-445b-a68f-e84d4fb5b4ad None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=94f2124d6d4e46ca9baa9728741b0e4c, role_id=d82425658c8545e6b546d601ef7b9b58)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:11.033726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2ef28-eeb5-445b-a68f-e84d4fb5b4ad None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:11.201374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 615/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:09:10 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/94f2124d6d4e46ca9baa9728741b0e4c/roles/d82425658c8545e6b546d601ef7b9b58 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:09:13.250020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8caa7-0aac-4e31-b159-1d38e0cf56fa None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:13.250584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8caa7-0aac-4e31-b159-1d38e0cf56fa None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:13.250584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8caa7-0aac-4e31-b159-1d38e0cf56fa None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:13.251985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 618/1233] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:13.257116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada0cab-a798-4105-891c-52c5257c8f2d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:13.257534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada0cab-a798-4105-891c-52c5257c8f2d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:13.257648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada0cab-a798-4105-891c-52c5257c8f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:13.273907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0ada0cab-a798-4105-891c-52c5257c8f2d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:13.275070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0ada0cab-a798-4105-891c-52c5257c8f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:13.339428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 616/1234] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:13.363367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8caa7-0aac-4e31-b159-1d38e0cf56fa None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:13.387734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:13.461444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8caa7-0aac-4e31-b159-1d38e0cf56fa None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:13.492997 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359e0c-066e-46dd-b358-7a342b3021d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:13.493770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359e0c-066e-46dd-b358-7a342b3021d7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:13.494050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359e0c-066e-46dd-b358-7a342b3021d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:13.494320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359e0c-066e-46dd-b358-7a342b3021d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:13.505558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50359e0c-066e-46dd-b358-7a342b3021d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-50359e0c-066e-46dd-b358-7a342b3021d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:09:13.513263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:13.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:13.515179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 619/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Mon Jul 11 11:09:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:13.527261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada0cab-a798-4105-891c-52c5257c8f2d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:13.565184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:13.615561 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada0cab-a798-4105-891c-52c5257c8f2d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:13.647027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:13.647702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:13.647986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:13.648260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:13.648994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:13.649300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:13.650647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:13.658556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 617/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Mon Jul 11 11:09:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:09:13.665092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359e0c-066e-46dd-b358-7a342b3021d7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:13.748856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:13.764104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359e0c-066e-46dd-b358-7a342b3021d7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:13.796515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:13.797285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:13.797591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:13.797866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:13.808515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:13.809999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:13.826368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:13.827434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:13.828503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:13.828503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:13.828503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:13.828503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:13.828503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 620/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:13.833866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:13.931130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a07c6-f680-4a6e-8b93-0025658126d9 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:13.939959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:13.963660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449877df-bae2-438d-86fc-883978139fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:13.964318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449877df-bae2-438d-86fc-883978139fef None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:13.964603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449877df-bae2-438d-86fc-883978139fef None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:13.964880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449877df-bae2-438d-86fc-883978139fef None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:13.968094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449877df-bae2-438d-86fc-883978139fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:13.968094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449877df-bae2-438d-86fc-883978139fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:13.968094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449877df-bae2-438d-86fc-883978139fef None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:13.978545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 618/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:13.984268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:14.084598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a2f4-d9b7-468d-8151-990fb7bf9203 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:14.117097 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:14.117884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:14.118166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:14.118438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:14.121226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:14.126236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:14.127710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:09:14.142059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:14.143183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:14.144623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 621/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:14.149055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449877df-bae2-438d-86fc-883978139fef None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:14.246868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449877df-bae2-438d-86fc-883978139fef None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:14.278960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:14.279674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:14.279958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:14.280236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:14.281062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:14.281375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:14.282780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:14.300492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 619/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 11 11:09:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:14.306199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:14.402600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff3358-018c-49d4-98a7-72b7604c5ff2 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:14.434839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cdd63-c72e-4d68-9a0b-fc51f024f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:14.435509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cdd63-c72e-4d68-9a0b-fc51f024f776 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:14.435825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cdd63-c72e-4d68-9a0b-fc51f024f776 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:14.436139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cdd63-c72e-4d68-9a0b-fc51f024f776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:14.497010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:14.503294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cdd63-c72e-4d68-9a0b-fc51f024f776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:14.505275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119cdd63-c72e-4d68-9a0b-fc51f024f776 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:14.602234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 622/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:09:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:09:14.608030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:14.635414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:14.704435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36878b37-df1e-44fd-b03d-715030c78bc5 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:14.736708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:14.737494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:14.737773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:14.738046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:14.748194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:14.749562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:14.766898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:14.768100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:14.769172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:14.769172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:14.769172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:14.769172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:14.769172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 620/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:14.773907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cdd63-c72e-4d68-9a0b-fc51f024f776 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:14.867687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:14.873382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119cdd63-c72e-4d68-9a0b-fc51f024f776 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:14.913028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:14.913693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:14.913975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:14.914251 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:14.914972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:14.915300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:14.916657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:14.928238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 623/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:14.933899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:15.015636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:15.036243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9d091-1be3-4d7f-a99a-69cdd75dda97 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:15.068955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:15.069784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:15.070061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:15.070335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:15.076847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:15.078266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:09:15.093076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:15.094166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:15.095406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:09:15.095406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:09:15.095406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:09:15.095406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:09:15.095406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:09:15.095406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:09:15.095406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:15.095406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:15.098927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 621/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:15.100150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:15.197652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e1861-f961-4d99-aef6-6e67348308d7 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:15.230625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc63f3e-e385-42af-9df8-a768bb6f28ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:15.231296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc63f3e-e385-42af-9df8-a768bb6f28ba None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:15.231579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc63f3e-e385-42af-9df8-a768bb6f28ba None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:15.231861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc63f3e-e385-42af-9df8-a768bb6f28ba None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:15.232641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc63f3e-e385-42af-9df8-a768bb6f28ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:15.232953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc63f3e-e385-42af-9df8-a768bb6f28ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:15.234361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc63f3e-e385-42af-9df8-a768bb6f28ba None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:15.246537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 624/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:15.252496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:15.349991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941c5a1-9237-488e-a89b-29d6a406abf6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:15.381530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7a145-2ac0-4a79-a133-5ff7923c76c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:15.382208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a145-2ac0-4a79-a133-5ff7923c76c0 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:15.382488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a145-2ac0-4a79-a133-5ff7923c76c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:15.382763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a145-2ac0-4a79-a133-5ff7923c76c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:15.383612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7a145-2ac0-4a79-a133-5ff7923c76c0 None admin] RBAC: Adding query filter params (role.id=d82425658c8545e6b546d601ef7b9b58, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=94f2124d6d4e46ca9baa9728741b0e4c) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:15.383924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7a145-2ac0-4a79-a133-5ff7923c76c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:15.385366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7a145-2ac0-4a79-a133-5ff7923c76c0 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:15.390997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 622/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:09:15 2022] GET /identity/v3/role_assignments?user.id=94f2124d6d4e46ca9baa9728741b0e4c&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=d82425658c8545e6b546d601ef7b9b58 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:17.446163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9576a3-54ea-46c9-9d93-cd851f66149a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:17.446696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9576a3-54ea-46c9-9d93-cd851f66149a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:17.446696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9576a3-54ea-46c9-9d93-cd851f66149a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:17.448133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 625/1247] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:17.453128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd8f8-f2d5-4e32-ac87-4b0e77f6f2c9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:17.453544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd8f8-f2d5-4e32-ac87-4b0e77f6f2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:17.453678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bd8f8-f2d5-4e32-ac87-4b0e77f6f2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:17.459386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:17.470967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bf7bd8f8-f2d5-4e32-ac87-4b0e77f6f2c9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:17.472155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bf7bd8f8-f2d5-4e32-ac87-4b0e77f6f2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:17.536061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 623/1248] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:17.559615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9576a3-54ea-46c9-9d93-cd851f66149a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:17.577915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:17.657969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9576a3-54ea-46c9-9d93-cd851f66149a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:17.690808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34216c43-5b3f-47c6-8ad7-712aff3ec365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:17.691652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34216c43-5b3f-47c6-8ad7-712aff3ec365 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:17.691986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34216c43-5b3f-47c6-8ad7-712aff3ec365 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:17.692269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34216c43-5b3f-47c6-8ad7-712aff3ec365 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:17.702101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34216c43-5b3f-47c6-8ad7-712aff3ec365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-34216c43-5b3f-47c6-8ad7-712aff3ec365 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:09:17.710074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:17.711202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:17.712118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 626/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:09:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:17.717219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd8f8-f2d5-4e32-ac87-4b0e77f6f2c9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:17.811193 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bd8f8-f2d5-4e32-ac87-4b0e77f6f2c9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:17.839521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:17.844233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:17.844888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:17.845169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:17.845443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:17.846176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:17.846481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:17.848556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:17.856185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 624/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:09:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:17.861845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34216c43-5b3f-47c6-8ad7-712aff3ec365 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:17.942007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:17.964396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34216c43-5b3f-47c6-8ad7-712aff3ec365 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:17.995398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:17.996176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:17.996480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:17.996754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:18.007269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:18.008777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:18.025849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:18.026901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:18.028180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:18.028180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:18.028180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:18.028180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:18.028180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 627/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:18.032840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:18.132152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf94b4a-8878-479d-bc1d-c11bf7ab9a7b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:18.164303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:18.164961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:18.165247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:18.165525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:18.166241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:18.166553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:18.167944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:18.180083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 625/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:18.185861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:18.293362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162dfbb-3e43-48e9-ba44-9a4bbd1a008a None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:18.319022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:18.326198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:18.326984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:18.327278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:18.327573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:18.335213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:18.336671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:09:18.351192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:18.352439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:18.353621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 628/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:18.358185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:18.410955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:18.453916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ab166-8eb0-4ccd-a022-24ecae940199 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:18.486055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:18.486721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:18.487007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:18.487309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:18.488631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:18.488944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:18.490361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:18.507576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 626/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 11 11:09:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:18.513374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:18.613932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520269cd-0060-472f-b6f2-f407369d064b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:18.646007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa3052-d108-45a8-8c81-9c8e8a378a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:18.646639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3052-d108-45a8-8c81-9c8e8a378a5d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:18.646920 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3052-d108-45a8-8c81-9c8e8a378a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:18.647193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3052-d108-45a8-8c81-9c8e8a378a5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:18.700160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:18.711944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa3052-d108-45a8-8c81-9c8e8a378a5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:18.713444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa3052-d108-45a8-8c81-9c8e8a378a5d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:18.809475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 629/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:09:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:09:18.815195 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:18.912691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d75874-6bdb-4079-8b78-3366153c9634 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:18.926156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:18.946634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:18.952725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:18.952725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:18.952725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:18.964484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:18.966222 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:18.984457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:18.985517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:18.986841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:18.986841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:18.986841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:18.986841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:18.986841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 627/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:18.994688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa3052-d108-45a8-8c81-9c8e8a378a5d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:19.081068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:19.092892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa3052-d108-45a8-8c81-9c8e8a378a5d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:19.125309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:19.125994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:19.126273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:19.126553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:19.127272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:19.127597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:19.128930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:19.139603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 630/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:19.145908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:19.245659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7e999-b294-430d-a25b-f2574be1920b None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:19.278006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:19.278823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:19.279107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:19.279410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:19.286294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:19.287748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:09:19.301470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:19.302518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:19.303657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 628/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:09:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:19.308403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:19.403569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947f306-f654-4fd0-b04a-aec90dd57975 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:19.435770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b46f-cbd0-4b2c-8531-500c2b5d3604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:19.436419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b46f-cbd0-4b2c-8531-500c2b5d3604 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:19.436698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b46f-cbd0-4b2c-8531-500c2b5d3604 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:19.436970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b46f-cbd0-4b2c-8531-500c2b5d3604 None admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:19.437739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b46f-cbd0-4b2c-8531-500c2b5d3604 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:19.438044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b46f-cbd0-4b2c-8531-500c2b5d3604 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:19.439468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b46f-cbd0-4b2c-8531-500c2b5d3604 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:19.452093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 631/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:09:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:19.458778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:19.484088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:19.554180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dbea9-e2a1-4d32-8292-c4255ae91022 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:19.587483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718d409-783b-4ab8-93f8-4069792bad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:19.588123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d409-783b-4ab8-93f8-4069792bad46 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:19.588403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d409-783b-4ab8-93f8-4069792bad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:19.588675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718d409-783b-4ab8-93f8-4069792bad46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:19.589512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718d409-783b-4ab8-93f8-4069792bad46 None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=94f2124d6d4e46ca9baa9728741b0e4c) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:19.589821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718d409-783b-4ab8-93f8-4069792bad46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:19.591286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718d409-783b-4ab8-93f8-4069792bad46 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:19.596633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 629/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:09:19 2022] GET /identity/v3/role_assignments?user.id=94f2124d6d4e46ca9baa9728741b0e4c&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=007af6bd1b734517b934b4cac45faf2c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:21.606378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeec794-05cf-48b3-959a-89af502dec4d None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:21.606924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeec794-05cf-48b3-959a-89af502dec4d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:21.606924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeec794-05cf-48b3-959a-89af502dec4d None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:21.608382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 632/1261] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:21.614132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dba5a-a3cd-4c5e-b998-766801c265be None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:21.614517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dba5a-a3cd-4c5e-b998-766801c265be None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:21.614681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dba5a-a3cd-4c5e-b998-766801c265be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:21.632062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-154dba5a-a3cd-4c5e-b998-766801c265be None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:21.633291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-154dba5a-a3cd-4c5e-b998-766801c265be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:21.698204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 630/1262] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:21.720557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeec794-05cf-48b3-959a-89af502dec4d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:21.731804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:21.819806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeec794-05cf-48b3-959a-89af502dec4d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:21.852734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ca776-5bcd-47f4-9e2a-cddc5e218408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:21.853510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ca776-5bcd-47f4-9e2a-cddc5e218408 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:21.853792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ca776-5bcd-47f4-9e2a-cddc5e218408 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:21.854066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ca776-5bcd-47f4-9e2a-cddc5e218408 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:21.865075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ca776-5bcd-47f4-9e2a-cddc5e218408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-a26ca776-5bcd-47f4-9e2a-cddc5e218408 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:09:21.872685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:21.873853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:21.874984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 633/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:09:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:21.880210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154dba5a-a3cd-4c5e-b998-766801c265be None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:21.898365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:21.980110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154dba5a-a3cd-4c5e-b998-766801c265be None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:22.013466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:22.014129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:22.014410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:22.014685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:22.015448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:22.015796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:22.017149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:22.025457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 631/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:09:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:22.031217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ca776-5bcd-47f4-9e2a-cddc5e218408 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:22.096378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:22.129298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ca776-5bcd-47f4-9e2a-cddc5e218408 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:22.161263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:22.162033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:22.162313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:22.162583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:22.173146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:22.174607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:22.191469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:22.192498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:22.193447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:22.193447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:22.193447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:22.193447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:22.193447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 634/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:22.198494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:22.283401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:22.301867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382022b1-d3f1-4ef4-a9d8-12e8c29e9597 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:22.334317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:22.334969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:22.335274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:22.335568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:22.336262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:22.336568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:22.337930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:22.349158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 632/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:22.354939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:22.453045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df1937-ff0d-45ac-bee6-88f9454eb1df None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:22.472532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:22.485836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:22.486627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:22.486909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:22.487198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:22.494540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:22.495954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:09:22.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:22.511562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:22.512710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 635/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:22.517519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:22.621505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e3e50-132d-441d-9d31-7f754ae233db None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:22.654283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:22.654949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:22.655235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:22.655592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:22.656442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:22.656733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:22.658235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:22.667093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:22.675028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 633/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 11 11:09:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:22.680723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:22.778711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf366b17-6d0a-435f-ae36-ff946107235b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:22.812274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:22.813054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:22.813336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:22.813612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:22.825240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:22.826747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:22.844186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:22.848145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:22.849341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:22.849341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:22.849341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:22.849341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:22.849341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 636/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:22.851691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:22.949212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c16d3b-5ebe-4b65-a0f9-3c1f0115b6f2 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:22.983514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:22.984229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:22.984518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:22.984801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:22.985525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:22.985838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:22.987212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:22.998309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 634/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:23.004140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:23.042259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:23.109231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a1836-9bb2-4e62-9300-cac632327456 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:23.141687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:23.142464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:23.142744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:23.143017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:23.150059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:23.151510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:09:23.166700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:23.167647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:23.168684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 637/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:23.174164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:23.234070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:23.274159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af14bd-cdca-451a-a631-25c4b1e816a1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:23.307152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b07cf0-1e2f-4f8c-847a-675647c62d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:23.307787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b07cf0-1e2f-4f8c-847a-675647c62d1d None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:23.308072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b07cf0-1e2f-4f8c-847a-675647c62d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:23.308352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b07cf0-1e2f-4f8c-847a-675647c62d1d None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:23.309133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b07cf0-1e2f-4f8c-847a-675647c62d1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:23.309446 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b07cf0-1e2f-4f8c-847a-675647c62d1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:23.310864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b07cf0-1e2f-4f8c-847a-675647c62d1d None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:23.322824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 635/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:09:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:23.328833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:23.416772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:23.425926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c40d0c-c303-45c4-a8ee-d39e4ab182a9 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:23.457893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73367831-23cd-4759-b60e-457e567f9ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:23.458772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367831-23cd-4759-b60e-457e567f9ff8 None admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:23.459054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367831-23cd-4759-b60e-457e567f9ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:23.459364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367831-23cd-4759-b60e-457e567f9ff8 None admin] PATH_INFO: `/v3/projects/a099df6dc96f4edeb762df2a36068788/users/94f2124d6d4e46ca9baa9728741b0e4c/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:23.461331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73367831-23cd-4759-b60e-457e567f9ff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a099df6dc96f4edeb762df2a36068788, user_id=94f2124d6d4e46ca9baa9728741b0e4c, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:23.462767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73367831-23cd-4759-b60e-457e567f9ff8 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:23.548842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 638/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Mon Jul 11 11:09:23 2022] PUT /identity/v3/projects/a099df6dc96f4edeb762df2a36068788/users/94f2124d6d4e46ca9baa9728741b0e4c/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:09:25.538020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0fac-9be7-4a2f-b2bb-16dbfb3efe2b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:25.538597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0fac-9be7-4a2f-b2bb-16dbfb3efe2b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:25.538597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0fac-9be7-4a2f-b2bb-16dbfb3efe2b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:25.539976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 636/1274] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:25.545060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7d767-aeec-4bbb-b9ef-57f2e22f7227 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:25.545702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7d767-aeec-4bbb-b9ef-57f2e22f7227 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:25.545883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7d767-aeec-4bbb-b9ef-57f2e22f7227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:25.562041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a6b7d767-aeec-4bbb-b9ef-57f2e22f7227 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:25.563298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a6b7d767-aeec-4bbb-b9ef-57f2e22f7227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:25.640177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 639/1275] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:25.664338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e0fac-9be7-4a2f-b2bb-16dbfb3efe2b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:25.764017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e0fac-9be7-4a2f-b2bb-16dbfb3efe2b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:25.780702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:25.797556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7b0b3-da77-4364-b583-4519540a7cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:25.798263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7b0b3-da77-4364-b583-4519540a7cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:25.798545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7b0b3-da77-4364-b583-4519540a7cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:25.798820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7b0b3-da77-4364-b583-4519540a7cd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:25.809503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7b0b3-da77-4364-b583-4519540a7cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-68b7b0b3-da77-4364-b583-4519540a7cd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 11:09:25.817228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:25.818361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:25.819452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 637/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Mon Jul 11 11:09:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:25.824637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7d767-aeec-4bbb-b9ef-57f2e22f7227 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:25.910720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:25.928199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7d767-aeec-4bbb-b9ef-57f2e22f7227 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:25.960880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:25.961547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:25.961826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:25.962101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:25.962847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:25.963158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:25.964556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:25.973496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 640/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Mon Jul 11 11:09:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:25.979447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7b0b3-da77-4364-b583-4519540a7cd6 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:26.075970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7b0b3-da77-4364-b583-4519540a7cd6 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:26.108748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:26.109663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:26.109944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:26.110219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:26.121190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:26.122619 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:26.141448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:26.146611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:26.147704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:26.147704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:26.147704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:26.147704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:26.147704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 638/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:09:26.148585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:26.254744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de7fe0-e5a6-4ff6-8db1-20819307bc70 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:26.287949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:26.288605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:26.288886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:26.289158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:26.289952 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:26.290255 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:26.291659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:26.304238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 641/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:26.309899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:26.339840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:26.406478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70091f2-1a83-41d4-b416-4ef976320325 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:26.440028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:26.440866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:26.441150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:26.441426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:26.448893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:26.450364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 11:09:26.464163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:26.465148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:26.466166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 639/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 11:09:26.471244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:26.475656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:26.567973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae4c3c-80d1-42b4-b3db-19166157aac0 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:26.602871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:26.603493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:26.603777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:26.604133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:26.604963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:26.605270 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:26.606761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:26.623847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 642/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Mon Jul 11 11:09:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:26.629750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:26.723322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:26.732382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463a3b9-85b4-4b44-84ff-4b65a7047aae None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:26.764566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cdb08-8cce-4793-b23b-22d8d9db6198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:26.765391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cdb08-8cce-4793-b23b-22d8d9db6198 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:26.765682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cdb08-8cce-4793-b23b-22d8d9db6198 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:26.766125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cdb08-8cce-4793-b23b-22d8d9db6198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:26.832257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cdb08-8cce-4793-b23b-22d8d9db6198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:26.833714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cdb08-8cce-4793-b23b-22d8d9db6198 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:26.922648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:26.933232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 640/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Mon Jul 11 11:09:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 11:09:26.939129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:26.998757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:27.037128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f8a16-20bf-4d2d-af52-93181605ff9d None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:27.071867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:27.072649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:27.072990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:27.073265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:27.083944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:27.085382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 11:09:27.102334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:27.103432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:27.104558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 11:09:27.104558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 11:09:27.104558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 11:09:27.104558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:27.104558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 643/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:27.109318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cdb08-8cce-4793-b23b-22d8d9db6198 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:27.207319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cdb08-8cce-4793-b23b-22d8d9db6198 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:27.240260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:27.240920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:27.248130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:27.248130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:27.248130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:27.248130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:27.248130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:27.255167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 641/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Mon Jul 11 11:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:27.261013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:27.361089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8d3aa-eeaf-457e-a86c-4b208a55cd63 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:27.374420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:27.393240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:27.394518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:27.394800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:27.395075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:27.401649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:27.403079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 11:09:27.417424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:27.418499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: ERROR keystone.server.flask.application Jul 11 11:09:27.419644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 644/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Mon Jul 11 11:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:27.424649 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:27.474895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:27.522870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cd6f1-bc3b-46ff-978c-196bd7ae0ed1 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:27.563603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b6bb4-5fff-4252-a808-dba9d2599e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:27.564273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6bb4-5fff-4252-a808-dba9d2599e2c None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:27.564558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6bb4-5fff-4252-a808-dba9d2599e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:27.564835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b6bb4-5fff-4252-a808-dba9d2599e2c None admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:27.565630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b6bb4-5fff-4252-a808-dba9d2599e2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:27.565940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b6bb4-5fff-4252-a808-dba9d2599e2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:27.567409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b6bb4-5fff-4252-a808-dba9d2599e2c None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:27.581828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 642/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Mon Jul 11 11:09:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:27.588232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:27.687045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7479c5c-5215-43ce-9cb1-43dee4e7a40b None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:27.720284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef31841-87b4-419a-be03-72dab08472ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:27.720923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef31841-87b4-419a-be03-72dab08472ce None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:27.721203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef31841-87b4-419a-be03-72dab08472ce None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:27.721476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef31841-87b4-419a-be03-72dab08472ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:27.722800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef31841-87b4-419a-be03-72dab08472ce None admin] RBAC: Adding query filter params (role.id=007af6bd1b734517b934b4cac45faf2c, scope.project.id=a099df6dc96f4edeb762df2a36068788, user.id=94f2124d6d4e46ca9baa9728741b0e4c) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:27.723105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef31841-87b4-419a-be03-72dab08472ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:27.724539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef31841-87b4-419a-be03-72dab08472ce None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:27.730175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 645/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Mon Jul 11 11:09:27 2022] GET /identity/v3/role_assignments?user.id=94f2124d6d4e46ca9baa9728741b0e4c&scope.project.id=a099df6dc96f4edeb762df2a36068788&role.id=007af6bd1b734517b934b4cac45faf2c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:29.785854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6d0fc-9c31-4cbb-a16c-5d3dde8add89 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:29.786393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6d0fc-9c31-4cbb-a16c-5d3dde8add89 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:29.786393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6d0fc-9c31-4cbb-a16c-5d3dde8add89 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:29.787814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 643/1288] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:29.792814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a72dbd-cc9b-439a-83f5-08546227cd92 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:29.793254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a72dbd-cc9b-439a-83f5-08546227cd92 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:29.793379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a72dbd-cc9b-439a-83f5-08546227cd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:29.810586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-29a72dbd-cc9b-439a-83f5-08546227cd92 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:29.811795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-29a72dbd-cc9b-439a-83f5-08546227cd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:29.840678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:29.877106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 646/1289] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:29.899823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6d0fc-9c31-4cbb-a16c-5d3dde8add89 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:29.996946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6d0fc-9c31-4cbb-a16c-5d3dde8add89 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:30.007488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:30.029534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:30.030330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:30.030616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:30.030893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:30.031576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:30.032942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:30.045370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:30.046638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:30.047733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 644/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:09:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:30.053195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a72dbd-cc9b-439a-83f5-08546227cd92 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:30.149114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a72dbd-cc9b-439a-83f5-08546227cd92 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:30.181457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de04b97-a9d4-43e2-8a4a-e47bc4c2ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:30.182109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de04b97-a9d4-43e2-8a4a-e47bc4c2ec05 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:30.182388 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de04b97-a9d4-43e2-8a4a-e47bc4c2ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:30.182659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de04b97-a9d4-43e2-8a4a-e47bc4c2ec05 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:30.183387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de04b97-a9d4-43e2-8a4a-e47bc4c2ec05 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:30.183744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de04b97-a9d4-43e2-8a4a-e47bc4c2ec05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:30.185136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de04b97-a9d4-43e2-8a4a-e47bc4c2ec05 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:30.189774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 647/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 11 11:09:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:30.196105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:30.294284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce59b3a-4ae0-4516-b787-98b9021c14ae None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:30.326841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a8aa8-cc2d-4ba9-bcc6-86f637ee93f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:30.327466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8aa8-cc2d-4ba9-bcc6-86f637ee93f6 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:30.327750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8aa8-cc2d-4ba9-bcc6-86f637ee93f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:30.328025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8aa8-cc2d-4ba9-bcc6-86f637ee93f6 None admin] PATH_INFO: `/v3/services` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:30.328721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a8aa8-cc2d-4ba9-bcc6-86f637ee93f6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:30.329030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a8aa8-cc2d-4ba9-bcc6-86f637ee93f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:30.330383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a8aa8-cc2d-4ba9-bcc6-86f637ee93f6 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:30.335189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 645/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 11 11:09:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:32.345729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd4e62-6f44-4bfb-a1f3-ec6fd89b3a76 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:32.346263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd4e62-6f44-4bfb-a1f3-ec6fd89b3a76 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:32.346263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd4e62-6f44-4bfb-a1f3-ec6fd89b3a76 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:32.348139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 648/1293] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:32.353261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a0f4b-9ef2-415a-8e76-fc0742412c8f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:32.353687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a0f4b-9ef2-415a-8e76-fc0742412c8f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:32.353838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a0f4b-9ef2-415a-8e76-fc0742412c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:32.370538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ef4a0f4b-9ef2-415a-8e76-fc0742412c8f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:32.371802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ef4a0f4b-9ef2-415a-8e76-fc0742412c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:32.399483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:32.435078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 646/1294] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:32.457966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd4e62-6f44-4bfb-a1f3-ec6fd89b3a76 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:32.489116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:32.557670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd4e62-6f44-4bfb-a1f3-ec6fd89b3a76 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:32.590835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187bcba-c2bd-48de-9c3f-21169b925c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:32.591513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187bcba-c2bd-48de-9c3f-21169b925c28 None admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:32.591789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187bcba-c2bd-48de-9c3f-21169b925c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:32.592058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187bcba-c2bd-48de-9c3f-21169b925c28 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:32.592766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187bcba-c2bd-48de-9c3f-21169b925c28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:32.594111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187bcba-c2bd-48de-9c3f-21169b925c28 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:32.702376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 649/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Mon Jul 11 11:09:32 2022] POST /identity/v3/services => generated 246 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:09:34.711658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c89b-111e-4ba6-b728-32325000a16c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:34.712239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c89b-111e-4ba6-b728-32325000a16c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:34.712239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7c89b-111e-4ba6-b728-32325000a16c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:34.713552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 647/1296] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:34.718590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36ff9c-9225-44c2-980a-991c0937d5e3 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:34.719028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36ff9c-9225-44c2-980a-991c0937d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:34.719155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36ff9c-9225-44c2-980a-991c0937d5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:34.736100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-eb36ff9c-9225-44c2-980a-991c0937d5e3 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:34.737873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-eb36ff9c-9225-44c2-980a-991c0937d5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:34.808883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 650/1297] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:34.831387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7c89b-111e-4ba6-b728-32325000a16c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:34.928779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7c89b-111e-4ba6-b728-32325000a16c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:34.960812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:34.961584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:34.961916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:34.962134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:34.962776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:34.964152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:34.976092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:34.987491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:34.988554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 648/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:09:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:34.989547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36ff9c-9225-44c2-980a-991c0937d5e3 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:35.001234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:35.083219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36ff9c-9225-44c2-980a-991c0937d5e3 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:35.116624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:35.117283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:35.117569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:35.117854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:35.118555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:35.118865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:35.120263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:35.125271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 651/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 11 11:09:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:35.131353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:35.163006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:35.231004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771c2b6-d0d3-4377-b2d3-3312bf3b5647 None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:35.263968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91becda-8c4e-45ed-bab4-f0da2e92b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:35.264616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91becda-8c4e-45ed-bab4-f0da2e92b18e None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:35.264898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91becda-8c4e-45ed-bab4-f0da2e92b18e None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:35.265170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91becda-8c4e-45ed-bab4-f0da2e92b18e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:35.265950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91becda-8c4e-45ed-bab4-f0da2e92b18e None admin] RBAC: Adding query filter params (interface=public, service_id=a4abacd1f4d8403db3a75722575fa3a1, region_id=RegionOne) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:35.266261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91becda-8c4e-45ed-bab4-f0da2e92b18e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:35.267732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91becda-8c4e-45ed-bab4-f0da2e92b18e None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:35.272780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 649/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Mon Jul 11 11:09:35 2022] GET /identity/v3/endpoints?service_id=a4abacd1f4d8403db3a75722575fa3a1&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:35.278614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:35.358524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:35.376438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f5454-9cd2-431e-916f-63417887f873 None admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:35.408411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c32eae-a352-45d1-a0b2-007b26b8e39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:35.409603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32eae-a352-45d1-a0b2-007b26b8e39e None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:35.409894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32eae-a352-45d1-a0b2-007b26b8e39e None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:35.410175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c32eae-a352-45d1-a0b2-007b26b8e39e None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:35.410916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c32eae-a352-45d1-a0b2-007b26b8e39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:35.412317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c32eae-a352-45d1-a0b2-007b26b8e39e None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:35.419528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 652/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Mon Jul 11 11:09:35 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:09:37.438576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77728f5-f14b-465a-b180-4f83a1700716 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:37.439133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77728f5-f14b-465a-b180-4f83a1700716 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:37.439133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77728f5-f14b-465a-b180-4f83a1700716 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:37.440586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 650/1302] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:09:37.446163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b7b1d-cf9f-4e22-9f4b-80fbc2a7861d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:37.453084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b7b1d-cf9f-4e22-9f4b-80fbc2a7861d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:37.453084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b7b1d-cf9f-4e22-9f4b-80fbc2a7861d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:37.462738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a24b7b1d-cf9f-4e22-9f4b-80fbc2a7861d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:09:37.463958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a24b7b1d-cf9f-4e22-9f4b-80fbc2a7861d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:09:37.529408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 653/1303] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:09:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 11:09:37.552211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77728f5-f14b-465a-b180-4f83a1700716 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:37.649720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:37.656175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77728f5-f14b-465a-b180-4f83a1700716 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:37.688303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:37.689144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:37.689448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:37.689651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:37.690822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:37.692218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:09:37.703658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:09:37.705630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:09:37.708336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 651/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Mon Jul 11 11:09:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 11:09:37.711653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b7b1d-cf9f-4e22-9f4b-80fbc2a7861d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:37.786967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:37.809661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b7b1d-cf9f-4e22-9f4b-80fbc2a7861d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:37.842082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c24d72-b7dd-4d19-bd83-cb534a94a030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:37.842724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c24d72-b7dd-4d19-bd83-cb534a94a030 None admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:37.843004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c24d72-b7dd-4d19-bd83-cb534a94a030 None admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:37.843294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c24d72-b7dd-4d19-bd83-cb534a94a030 None admin] PATH_INFO: `/v3/services` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:37.844076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c24d72-b7dd-4d19-bd83-cb534a94a030 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:09:37.844379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c24d72-b7dd-4d19-bd83-cb534a94a030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:37.845716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c24d72-b7dd-4d19-bd83-cb534a94a030 None admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:37.851433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 654/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Mon Jul 11 11:09:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:09:37.857373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:09:37.954836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c32842-1f72-42d0-ba8c-e2574510923f None admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:09:37.987970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cd377-0959-4f18-b900-2c8a928b9c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:09:37.988624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cd377-0959-4f18-b900-2c8a928b9c3a None admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:09:37.988907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cd377-0959-4f18-b900-2c8a928b9c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:09:37.989185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cd377-0959-4f18-b900-2c8a928b9c3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:09:37.989926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cd377-0959-4f18-b900-2c8a928b9c3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:09:37.991294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cd377-0959-4f18-b900-2c8a928b9c3a None admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:09:38.103182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 652/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Mon Jul 11 11:09:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:09:47.930859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=30 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:09:48.098390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:07.108886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5477-9dd1-41e1-9444-ba29c5a04b86 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:07.109352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5477-9dd1-41e1-9444-ba29c5a04b86 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:07.110099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5477-9dd1-41e1-9444-ba29c5a04b86 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:07.113959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 655/1307] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:07.118753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0850-67d8-4ade-8dfc-2ea8dc1a83af None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:07.119143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0850-67d8-4ade-8dfc-2ea8dc1a83af None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:07.119386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d0850-67d8-4ade-8dfc-2ea8dc1a83af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:07.285001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-770d0850-67d8-4ade-8dfc-2ea8dc1a83af None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:07.286153 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-770d0850-67d8-4ade-8dfc-2ea8dc1a83af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:07.488236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 653/1308] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:07.537855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce7e26-ed0f-40b6-ad1d-de0202f99279 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:07.538395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce7e26-ed0f-40b6-ad1d-de0202f99279 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:07.538857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce7e26-ed0f-40b6-ad1d-de0202f99279 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:07.540976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 656/1309] 10.222.0.26 () {54 vars in 964 bytes} [Mon Jul 11 11:17:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:07.547941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd909d8-583a-4029-b63c-071a6c84721c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:07.548223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd909d8-583a-4029-b63c-071a6c84721c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:07.552655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd909d8-583a-4029-b63c-071a6c84721c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:07.629964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2dd909d8-583a-4029-b63c-071a6c84721c None None] MFA Rules not processed for user `cb1fa1e11b1a44aaa31cc19f7c199285`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:07.631067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2dd909d8-583a-4029-b63c-071a6c84721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:07.654222 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:07.724688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce7e26-ed0f-40b6-ad1d-de0202f99279 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:07.726400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-95ce7e26-ed0f-40b6-ad1d-de0202f99279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:07.730755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 654/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Jul 11 11:17:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:07.845970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce7e26-ed0f-40b6-ad1d-de0202f99279 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:07.938562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b645370-f165-47e3-942e-2520aa625eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:07.939326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b645370-f165-47e3-942e-2520aa625eed service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:07.939617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b645370-f165-47e3-942e-2520aa625eed service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:07.939963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b645370-f165-47e3-942e-2520aa625eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:07.941780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7b645370-f165-47e3-942e-2520aa625eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:08.009644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:08.075673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b645370-f165-47e3-942e-2520aa625eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:08.077457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b645370-f165-47e3-942e-2520aa625eed service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:08.098803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 657/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:10.472066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e76ed-3578-4180-8ff2-e7ed4f4e3df0 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:10.472643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e76ed-3578-4180-8ff2-e7ed4f4e3df0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:10.472643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e76ed-3578-4180-8ff2-e7ed4f4e3df0 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:10.473984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 655/1312] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:10.479210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b004-4bd5-41e1-8aaf-870fc89542c4 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:10.479680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b004-4bd5-41e1-8aaf-870fc89542c4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:10.479968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b004-4bd5-41e1-8aaf-870fc89542c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:10.548730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8a43b004-4bd5-41e1-8aaf-870fc89542c4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:10.549903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8a43b004-4bd5-41e1-8aaf-870fc89542c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:10.645862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 658/1313] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:10.683959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e76ed-3578-4180-8ff2-e7ed4f4e3df0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:10.685489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f44e76ed-3578-4180-8ff2-e7ed4f4e3df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:10.797845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e76ed-3578-4180-8ff2-e7ed4f4e3df0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:10.873346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:10.876221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fa9ce-4872-4211-b1e8-5ced5a9bc1d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:10.876872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fa9ce-4872-4211-b1e8-5ced5a9bc1d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:10.877156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fa9ce-4872-4211-b1e8-5ced5a9bc1d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:10.877433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fa9ce-4872-4211-b1e8-5ced5a9bc1d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:10.879054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cb6fa9ce-4872-4211-b1e8-5ced5a9bc1d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:10.988493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fa9ce-4872-4211-b1e8-5ced5a9bc1d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:10.989964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6fa9ce-4872-4211-b1e8-5ced5a9bc1d6 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:11.010462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 656/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:13.466792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524246-e67a-47ab-a559-7b7461b017d2 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:13.467386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524246-e67a-47ab-a559-7b7461b017d2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:13.467386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61524246-e67a-47ab-a559-7b7461b017d2 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:13.468783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 659/1315] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:13.474445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d7b5f-b893-47ae-95a9-1ff41cc84d7c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:13.474857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d7b5f-b893-47ae-95a9-1ff41cc84d7c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:13.475002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d7b5f-b893-47ae-95a9-1ff41cc84d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:13.537605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-804d7b5f-b893-47ae-95a9-1ff41cc84d7c None None] MFA Rules not processed for user `0d3b8b0ba0b84fecbef6319c3179a020`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:13.538819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-804d7b5f-b893-47ae-95a9-1ff41cc84d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:13.589101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:13.631144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 657/1316] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:13.670273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61524246-e67a-47ab-a559-7b7461b017d2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:13.671821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-61524246-e67a-47ab-a559-7b7461b017d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:13.681648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:13.771432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61524246-e67a-47ab-a559-7b7461b017d2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:13.840858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cb121-e962-4db0-bef6-8f9d31fc7a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:13.841486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb121-e962-4db0-bef6-8f9d31fc7a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:13.841764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb121-e962-4db0-bef6-8f9d31fc7a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:13.842036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cb121-e962-4db0-bef6-8f9d31fc7a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:13.843688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a49cb121-e962-4db0-bef6-8f9d31fc7a59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:13.902216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:13.947620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cb121-e962-4db0-bef6-8f9d31fc7a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:13.949040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cb121-e962-4db0-bef6-8f9d31fc7a59 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:13.966641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 660/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:16.762944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8dd92-2772-4f39-9df7-7b048872e0fa None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:16.763621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8dd92-2772-4f39-9df7-7b048872e0fa None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:16.763621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8dd92-2772-4f39-9df7-7b048872e0fa None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:16.764898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 658/1318] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:16.769915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941607-d33b-4cc4-8811-3fafbc4d4f65 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:16.770374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941607-d33b-4cc4-8811-3fafbc4d4f65 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:16.770483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941607-d33b-4cc4-8811-3fafbc4d4f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:16.832937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5c941607-d33b-4cc4-8811-3fafbc4d4f65 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:16.834111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5c941607-d33b-4cc4-8811-3fafbc4d4f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:16.927310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 661/1319] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:16.965898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8dd92-2772-4f39-9df7-7b048872e0fa None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:16.967449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-acf8dd92-2772-4f39-9df7-7b048872e0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:17.068490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8dd92-2772-4f39-9df7-7b048872e0fa None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:17.139881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d805f02-7418-418d-9fda-30fbc09e88b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:17.141027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805f02-7418-418d-9fda-30fbc09e88b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:17.141306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805f02-7418-418d-9fda-30fbc09e88b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:17.141578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d805f02-7418-418d-9fda-30fbc09e88b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:17.143196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9d805f02-7418-418d-9fda-30fbc09e88b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:17.151392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:17.248350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d805f02-7418-418d-9fda-30fbc09e88b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:17.249792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d805f02-7418-418d-9fda-30fbc09e88b2 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:17.267837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 659/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:20.285802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96925-b393-4068-8464-263bdcb07f7c None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:20.286375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96925-b393-4068-8464-263bdcb07f7c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:20.286375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f96925-b393-4068-8464-263bdcb07f7c None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:20.287743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 662/1321] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:20.292880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256953ac-6302-49cf-a67c-0eb133901cce None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:20.293333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256953ac-6302-49cf-a67c-0eb133901cce None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:20.293425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256953ac-6302-49cf-a67c-0eb133901cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:20.358557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-256953ac-6302-49cf-a67c-0eb133901cce None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:20.360337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-256953ac-6302-49cf-a67c-0eb133901cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:20.426341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:20.464713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 660/1322] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:17:20.501062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f96925-b393-4068-8464-263bdcb07f7c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:20.502662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d8f96925-b393-4068-8464-263bdcb07f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:20.521979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:20.609814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f96925-b393-4068-8464-263bdcb07f7c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:20.688047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56bb4c-8279-40b1-84be-4edfe3c98905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:20.688706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56bb4c-8279-40b1-84be-4edfe3c98905 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:20.688990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56bb4c-8279-40b1-84be-4edfe3c98905 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:20.689271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56bb4c-8279-40b1-84be-4edfe3c98905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:20.690969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8a56bb4c-8279-40b1-84be-4edfe3c98905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:20.768347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:20.808286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56bb4c-8279-40b1-84be-4edfe3c98905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:20.809744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56bb4c-8279-40b1-84be-4edfe3c98905 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:20.827380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 663/1323] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:22.954135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf41b-2831-4437-8c81-51df7c7a3654 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:22.954677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf41b-2831-4437-8c81-51df7c7a3654 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:22.954677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cf41b-2831-4437-8c81-51df7c7a3654 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:22.956117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 661/1324] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:22.961422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323c94-82ab-4e90-ab78-1e87f8ab2a87 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:22.961840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323c94-82ab-4e90-ab78-1e87f8ab2a87 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:22.961981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b323c94-82ab-4e90-ab78-1e87f8ab2a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:23.025397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8b323c94-82ab-4e90-ab78-1e87f8ab2a87 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:23.026590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8b323c94-82ab-4e90-ab78-1e87f8ab2a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:23.125289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 664/1325] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:23.160217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cf41b-2831-4437-8c81-51df7c7a3654 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:23.161723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c29cf41b-2831-4437-8c81-51df7c7a3654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:23.263458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cf41b-2831-4437-8c81-51df7c7a3654 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:23.335209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb3f9a-dd7a-46da-a670-aebb7c3431ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:23.335888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3f9a-dd7a-46da-a670-aebb7c3431ab service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:23.336175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3f9a-dd7a-46da-a670-aebb7c3431ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:23.336454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3f9a-dd7a-46da-a670-aebb7c3431ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:23.338105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-90bb3f9a-dd7a-46da-a670-aebb7c3431ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:23.355186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:23.447055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb3f9a-dd7a-46da-a670-aebb7c3431ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:23.448518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb3f9a-dd7a-46da-a670-aebb7c3431ab service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:23.466641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 662/1326] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:17:25.700522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be181060-051e-4714-9864-cb8de942b469 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:25.701085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be181060-051e-4714-9864-cb8de942b469 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:25.701085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be181060-051e-4714-9864-cb8de942b469 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:25.702472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 665/1327] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:17:25.707983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a521b8-e60c-44aa-a3cb-88ee62337f05 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:25.708396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a521b8-e60c-44aa-a3cb-88ee62337f05 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:25.708544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a521b8-e60c-44aa-a3cb-88ee62337f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:25.771526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c1a521b8-e60c-44aa-a3cb-88ee62337f05 None None] MFA Rules not processed for user `0d3b8b0ba0b84fecbef6319c3179a020`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:25.772732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c1a521b8-e60c-44aa-a3cb-88ee62337f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:25.837766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:25.870381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 663/1328] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:25.908217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be181060-051e-4714-9864-cb8de942b469 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:25.909768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-be181060-051e-4714-9864-cb8de942b469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:25.941067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:26.011778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be181060-051e-4714-9864-cb8de942b469 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:26.086038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1993443-29f5-4a4a-ba62-e2447099b062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:26.086693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1993443-29f5-4a4a-ba62-e2447099b062 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:26.086975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1993443-29f5-4a4a-ba62-e2447099b062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:26.087268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1993443-29f5-4a4a-ba62-e2447099b062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:26.089001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c1993443-29f5-4a4a-ba62-e2447099b062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:26.167612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:26.198774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1993443-29f5-4a4a-ba62-e2447099b062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:26.200504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1993443-29f5-4a4a-ba62-e2447099b062 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:26.219159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 666/1329] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:28.678607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b602d5-98f0-47f4-be86-09b898f9e86d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:28.679156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b602d5-98f0-47f4-be86-09b898f9e86d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:28.679156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b602d5-98f0-47f4-be86-09b898f9e86d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:28.680933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 664/1330] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:28.685956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb09d34-b19e-4f1f-a9ba-df3ad91e03b8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:28.686381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb09d34-b19e-4f1f-a9ba-df3ad91e03b8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:28.686563 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb09d34-b19e-4f1f-a9ba-df3ad91e03b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:28.750268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ffb09d34-b19e-4f1f-a9ba-df3ad91e03b8 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:28.751480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ffb09d34-b19e-4f1f-a9ba-df3ad91e03b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:28.844775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 667/1331] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:28.894371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81801c74-7015-429f-9ff6-2bbc846d7e28 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:28.894661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81801c74-7015-429f-9ff6-2bbc846d7e28 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:28.894945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81801c74-7015-429f-9ff6-2bbc846d7e28 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:28.896354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 665/1332] 10.222.0.26 () {54 vars in 964 bytes} [Mon Jul 11 11:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:28.902183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05206fb8-6f30-43f8-ae0f-6324fc90cf8a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:28.902499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05206fb8-6f30-43f8-ae0f-6324fc90cf8a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:28.902780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05206fb8-6f30-43f8-ae0f-6324fc90cf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:28.947023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:28.977163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-05206fb8-6f30-43f8-ae0f-6324fc90cf8a None None] MFA Rules not processed for user `cb1fa1e11b1a44aaa31cc19f7c199285`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:28.978365 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-05206fb8-6f30-43f8-ae0f-6324fc90cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:29.052655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 668/1333] 10.222.0.26 () {58 vars in 1061 bytes} [Mon Jul 11 11:17:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:29.061962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81801c74-7015-429f-9ff6-2bbc846d7e28 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:29.063493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-81801c74-7015-429f-9ff6-2bbc846d7e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:29.165723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81801c74-7015-429f-9ff6-2bbc846d7e28 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:29.240358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccdd82-fe6c-4a2d-af1d-368dd8848373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:29.241006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccdd82-fe6c-4a2d-af1d-368dd8848373 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:29.241287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccdd82-fe6c-4a2d-af1d-368dd8848373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:29.241562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccdd82-fe6c-4a2d-af1d-368dd8848373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:29.243215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-19ccdd82-fe6c-4a2d-af1d-368dd8848373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:29.270148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:29.352157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccdd82-fe6c-4a2d-af1d-368dd8848373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:29.353592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccdd82-fe6c-4a2d-af1d-368dd8848373 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:29.371060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 666/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:31.667991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d676ac6-2931-4fda-8eb2-405a1b5d5056 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:31.668572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d676ac6-2931-4fda-8eb2-405a1b5d5056 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:31.668572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d676ac6-2931-4fda-8eb2-405a1b5d5056 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:31.669929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 669/1335] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:31.675109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38715a24-19d8-44b4-99ec-42eff403d5b0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:31.675512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38715a24-19d8-44b4-99ec-42eff403d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:31.675713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38715a24-19d8-44b4-99ec-42eff403d5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:31.739145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-38715a24-19d8-44b4-99ec-42eff403d5b0 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:31.740382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-38715a24-19d8-44b4-99ec-42eff403d5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:31.811683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:31.834884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 667/1336] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:31.874008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d676ac6-2931-4fda-8eb2-405a1b5d5056 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:31.876117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5d676ac6-2931-4fda-8eb2-405a1b5d5056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:31.979694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d676ac6-2931-4fda-8eb2-405a1b5d5056 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:32.006817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:32.050591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8988a85-59cd-410e-87c8-73932202cc21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:32.051236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8988a85-59cd-410e-87c8-73932202cc21 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:32.051558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8988a85-59cd-410e-87c8-73932202cc21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:32.051835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8988a85-59cd-410e-87c8-73932202cc21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:32.053482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f8988a85-59cd-410e-87c8-73932202cc21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:32.159674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8988a85-59cd-410e-87c8-73932202cc21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:32.161098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8988a85-59cd-410e-87c8-73932202cc21 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:32.179535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 670/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:35.091583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08538301-9d97-4473-af7f-fa17d81b876a None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:35.092028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08538301-9d97-4473-af7f-fa17d81b876a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:35.092142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08538301-9d97-4473-af7f-fa17d81b876a None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:35.094187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 668/1338] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:35.099436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f848d-1ae4-4316-9fe5-c84a94396b18 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:35.099889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f848d-1ae4-4316-9fe5-c84a94396b18 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:35.099985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f848d-1ae4-4316-9fe5-c84a94396b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:35.163043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-147f848d-1ae4-4316-9fe5-c84a94396b18 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:35.164281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-147f848d-1ae4-4316-9fe5-c84a94396b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:35.172384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:35.259823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 671/1339] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:35.296519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08538301-9d97-4473-af7f-fa17d81b876a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:35.298020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-08538301-9d97-4473-af7f-fa17d81b876a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:35.398021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08538301-9d97-4473-af7f-fa17d81b876a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:35.467816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d2ff1-c945-4f7a-9883-ede7fef41cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:35.468563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d2ff1-c945-4f7a-9883-ede7fef41cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:35.468843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d2ff1-c945-4f7a-9883-ede7fef41cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:35.469115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d2ff1-c945-4f7a-9883-ede7fef41cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:35.470721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bc5d2ff1-c945-4f7a-9883-ede7fef41cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:35.504424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:35.574404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d2ff1-c945-4f7a-9883-ede7fef41cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:35.575854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d2ff1-c945-4f7a-9883-ede7fef41cdb service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:35.593978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 669/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:39.964955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5581e-45a8-4c4c-a802-a974917097b8 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:39.965512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5581e-45a8-4c4c-a802-a974917097b8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:39.965512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5581e-45a8-4c4c-a802-a974917097b8 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:39.967094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 672/1341] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:17:39.972796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f47415-486c-4282-af1f-82fcd0aa3542 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:39.973294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f47415-486c-4282-af1f-82fcd0aa3542 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:39.973406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f47415-486c-4282-af1f-82fcd0aa3542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:40.041867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-61f47415-486c-4282-af1f-82fcd0aa3542 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:40.043095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-61f47415-486c-4282-af1f-82fcd0aa3542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:40.133191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:40.143925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 670/1342] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:40.182791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5581e-45a8-4c4c-a802-a974917097b8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:40.184397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-10a5581e-45a8-4c4c-a802-a974917097b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:40.289197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5581e-45a8-4c4c-a802-a974917097b8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:40.328110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:40.364389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ab819-52c6-437d-8c71-874cd498315d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:40.365045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ab819-52c6-437d-8c71-874cd498315d service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:40.365329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ab819-52c6-437d-8c71-874cd498315d service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:40.365607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ab819-52c6-437d-8c71-874cd498315d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:40.367295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d99ab819-52c6-437d-8c71-874cd498315d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:40.478489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ab819-52c6-437d-8c71-874cd498315d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:40.479962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ab819-52c6-437d-8c71-874cd498315d service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:40.498697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 673/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:43.562343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f7c90-38c5-4fe1-9ccb-6cf19f39e90f None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:43.562863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f7c90-38c5-4fe1-9ccb-6cf19f39e90f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:43.562863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f7c90-38c5-4fe1-9ccb-6cf19f39e90f None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:43.564366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 671/1344] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:43.569420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2356e-1456-4e67-ad22-e57ffab47929 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:43.569810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2356e-1456-4e67-ad22-e57ffab47929 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:43.569983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2356e-1456-4e67-ad22-e57ffab47929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:43.633799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4ff2356e-1456-4e67-ad22-e57ffab47929 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:43.635003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4ff2356e-1456-4e67-ad22-e57ffab47929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:43.652407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:43.730320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 674/1345] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:43.768147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f7c90-38c5-4fe1-9ccb-6cf19f39e90f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:43.769794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a76f7c90-38c5-4fe1-9ccb-6cf19f39e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:43.870833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f7c90-38c5-4fe1-9ccb-6cf19f39e90f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:43.943524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f5ce9-b4c3-4041-bea2-b08a63304ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:43.944180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5ce9-b4c3-4041-bea2-b08a63304ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:43.944460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5ce9-b4c3-4041-bea2-b08a63304ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:43.944732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5ce9-b4c3-4041-bea2-b08a63304ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:43.946447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c12f5ce9-b4c3-4041-bea2-b08a63304ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:43.993767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:44.060514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5ce9-b4c3-4041-bea2-b08a63304ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:44.061982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5ce9-b4c3-4041-bea2-b08a63304ce0 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:44.082021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 672/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:48.390485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718165d8-1f69-4bfd-88b4-4422f77e5c19 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:48.391071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718165d8-1f69-4bfd-88b4-4422f77e5c19 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:48.391071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718165d8-1f69-4bfd-88b4-4422f77e5c19 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:48.392568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 675/1347] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:48.398025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac394550-30f3-4429-8183-745501b183a4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:48.404476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac394550-30f3-4429-8183-745501b183a4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:48.404476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac394550-30f3-4429-8183-745501b183a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:48.468252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ac394550-30f3-4429-8183-745501b183a4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:48.469458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ac394550-30f3-4429-8183-745501b183a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:48.578088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:48.580589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 673/1348] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:48.618998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718165d8-1f69-4bfd-88b4-4422f77e5c19 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:17:48.620651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-718165d8-1f69-4bfd-88b4-4422f77e5c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:48.726674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718165d8-1f69-4bfd-88b4-4422f77e5c19 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:17:48.779328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:48.805352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbfa91-b081-4b5f-bbcb-2e159747d6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:17:48.805986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfa91-b081-4b5f-bbcb-2e159747d6ef service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:48.806267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfa91-b081-4b5f-bbcb-2e159747d6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:48.806539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfa91-b081-4b5f-bbcb-2e159747d6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:48.808222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-05cbfa91-b081-4b5f-bbcb-2e159747d6ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:48.913726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbfa91-b081-4b5f-bbcb-2e159747d6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:17:48.915151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbfa91-b081-4b5f-bbcb-2e159747d6ef service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:17:48.933601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 676/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:17:53.815916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d08c9d-9f7a-4487-a6a4-38b655ed312e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:53.816389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d08c9d-9f7a-4487-a6a4-38b655ed312e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:53.816487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d08c9d-9f7a-4487-a6a4-38b655ed312e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:53.817996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 674/1350] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:53.823468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00a70-b3a1-49b2-9494-bbea2c08379d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:53.824220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00a70-b3a1-49b2-9494-bbea2c08379d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:53.824651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f00a70-b3a1-49b2-9494-bbea2c08379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:53.965837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-92f00a70-b3a1-49b2-9494-bbea2c08379d None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:53.967395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-92f00a70-b3a1-49b2-9494-bbea2c08379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:53.988384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:54.090395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 677/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:17:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:58.187191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c3d3-6ce4-468b-b5c6-e30c125d2f24 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:58.187963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c3d3-6ce4-468b-b5c6-e30c125d2f24 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:58.188448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c3d3-6ce4-468b-b5c6-e30c125d2f24 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:58.189888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 675/1352] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:58.195025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e1198-1dbd-4685-a4ab-b21ffff6af0b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:58.195462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e1198-1dbd-4685-a4ab-b21ffff6af0b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:58.195673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e1198-1dbd-4685-a4ab-b21ffff6af0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:58.272785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-606e1198-1dbd-4685-a4ab-b21ffff6af0b None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:58.274019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-606e1198-1dbd-4685-a4ab-b21ffff6af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:58.351523 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:17:58.387562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 678/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:17:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:17:59.731823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269d201-b5f1-4add-b81b-0d367d0a5b31 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:59.732386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269d201-b5f1-4add-b81b-0d367d0a5b31 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:59.732386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269d201-b5f1-4add-b81b-0d367d0a5b31 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:59.733842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 676/1354] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:17:59.740446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87900c2e-cadd-41b3-86f2-f7f93c420c62 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:17:59.741504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87900c2e-cadd-41b3-86f2-f7f93c420c62 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:17:59.741940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87900c2e-cadd-41b3-86f2-f7f93c420c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:17:59.818170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-87900c2e-cadd-41b3-86f2-f7f93c420c62 None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:17:59.819407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-87900c2e-cadd-41b3-86f2-f7f93c420c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:17:59.927385 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 679/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:03.486941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddaa41-a701-44a6-b6b6-55c2e9cb413f None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:03.487520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddaa41-a701-44a6-b6b6-55c2e9cb413f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:03.487520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddaa41-a701-44a6-b6b6-55c2e9cb413f None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:03.488808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 677/1356] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:03.494198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2fdc2-0fd6-4cba-8715-3c9bf54ef110 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:03.494654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2fdc2-0fd6-4cba-8715-3c9bf54ef110 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:03.494764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2fdc2-0fd6-4cba-8715-3c9bf54ef110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:03.530895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:03.564548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9bf2fdc2-0fd6-4cba-8715-3c9bf54ef110 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:03.565738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9bf2fdc2-0fd6-4cba-8715-3c9bf54ef110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:03.667235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 680/1357] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:03.722397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efe412-f493-4924-bc32-f3cb96eff051 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:03.723354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efe412-f493-4924-bc32-f3cb96eff051 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:03.723793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efe412-f493-4924-bc32-f3cb96eff051 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:03.727486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 678/1358] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 11 11:18:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:03.737615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b0be-44fb-4569-80be-6b796b8836e8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:03.745866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b0be-44fb-4569-80be-6b796b8836e8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:03.745866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b0be-44fb-4569-80be-6b796b8836e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:03.829039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-df93b0be-44fb-4569-80be-6b796b8836e8 None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:03.830414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-df93b0be-44fb-4569-80be-6b796b8836e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:03.882834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:03.961603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 681/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 11 11:18:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:03.962321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efe412-f493-4924-bc32-f3cb96eff051 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:03.963851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c0efe412-f493-4924-bc32-f3cb96eff051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:04.112297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efe412-f493-4924-bc32-f3cb96eff051 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:04.188140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:04.231126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba624f-1dac-40ba-adc6-f71c19ec5d3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:04.231794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba624f-1dac-40ba-adc6-f71c19ec5d3b service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:04.232182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba624f-1dac-40ba-adc6-f71c19ec5d3b service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:04.232583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba624f-1dac-40ba-adc6-f71c19ec5d3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:04.234356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-53ba624f-1dac-40ba-adc6-f71c19ec5d3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:04.339035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba624f-1dac-40ba-adc6-f71c19ec5d3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:04.340842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba624f-1dac-40ba-adc6-f71c19ec5d3b service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:04.365567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 679/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:18:06.621535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b989-f947-45fe-a68f-d0f69f9e7b15 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:06.622055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b989-f947-45fe-a68f-d0f69f9e7b15 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:06.622244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738b989-f947-45fe-a68f-d0f69f9e7b15 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:06.624501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 682/1361] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:06.629698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413c359-5015-4d31-97f5-126e865cd0f2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:06.630149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413c359-5015-4d31-97f5-126e865cd0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:06.630246 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9413c359-5015-4d31-97f5-126e865cd0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:06.702139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9413c359-5015-4d31-97f5-126e865cd0f2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:06.703529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9413c359-5015-4d31-97f5-126e865cd0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:06.742341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:06.807067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 680/1362] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:06.855036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738b989-f947-45fe-a68f-d0f69f9e7b15 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:06.856762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7738b989-f947-45fe-a68f-d0f69f9e7b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:07.024018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738b989-f947-45fe-a68f-d0f69f9e7b15 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:07.034617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:07.161820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed03a6-f533-4017-8805-7fcb9d8947c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:07.162474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed03a6-f533-4017-8805-7fcb9d8947c5 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:07.162758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed03a6-f533-4017-8805-7fcb9d8947c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:07.163108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed03a6-f533-4017-8805-7fcb9d8947c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:07.164792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1bed03a6-f533-4017-8805-7fcb9d8947c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:07.265169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed03a6-f533-4017-8805-7fcb9d8947c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:07.266791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed03a6-f533-4017-8805-7fcb9d8947c5 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:07.289897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 683/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:09.805972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a8bff-b692-4d14-be85-ff311298082b None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:09.806539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a8bff-b692-4d14-be85-ff311298082b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:09.806539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a8bff-b692-4d14-be85-ff311298082b None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:09.808236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 681/1364] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:09.813485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bc366-1b9d-4f1b-88f2-6d35fef613cd None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:09.813970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bc366-1b9d-4f1b-88f2-6d35fef613cd None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:09.814068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bc366-1b9d-4f1b-88f2-6d35fef613cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:09.829534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:09.883730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-470bc366-1b9d-4f1b-88f2-6d35fef613cd None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:09.884958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-470bc366-1b9d-4f1b-88f2-6d35fef613cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:09.985576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 684/1365] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:10.032289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf842-cb7c-404a-986d-4fe520caa666 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:10.032665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf842-cb7c-404a-986d-4fe520caa666 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:10.033280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf842-cb7c-404a-986d-4fe520caa666 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:10.034676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 682/1366] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 11 11:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:10.040375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62651440-f4df-4774-b502-98d4a422d677 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:10.040680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62651440-f4df-4774-b502-98d4a422d677 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:10.044873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62651440-f4df-4774-b502-98d4a422d677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:10.128532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-62651440-f4df-4774-b502-98d4a422d677 None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:10.129748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-62651440-f4df-4774-b502-98d4a422d677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:10.167463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:10.258712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 685/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 11 11:18:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:10.266817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cf842-cb7c-404a-986d-4fe520caa666 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:10.268514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d21cf842-cb7c-404a-986d-4fe520caa666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:10.424958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cf842-cb7c-404a-986d-4fe520caa666 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:10.446063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:10.555415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e7bf6-89e8-413d-9523-fd44e47b2469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:10.556075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7bf6-89e8-413d-9523-fd44e47b2469 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:10.556364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7bf6-89e8-413d-9523-fd44e47b2469 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:10.556646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e7bf6-89e8-413d-9523-fd44e47b2469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:10.558325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-263e7bf6-89e8-413d-9523-fd44e47b2469 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:10.653107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e7bf6-89e8-413d-9523-fd44e47b2469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:10.654746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e7bf6-89e8-413d-9523-fd44e47b2469 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:10.674686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 683/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:11.656895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48c52e-7eda-4f29-8c60-abb1a5fce9d6 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:11.657457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48c52e-7eda-4f29-8c60-abb1a5fce9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:11.657457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48c52e-7eda-4f29-8c60-abb1a5fce9d6 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:11.658978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 686/1369] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:11.663999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab2511-2262-4931-b078-32f7cbf12e61 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:11.664463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab2511-2262-4931-b078-32f7cbf12e61 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:11.664555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab2511-2262-4931-b078-32f7cbf12e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:11.696536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:11.731233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fdab2511-2262-4931-b078-32f7cbf12e61 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:11.733552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fdab2511-2262-4931-b078-32f7cbf12e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:11.831814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 684/1370] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:12.173129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a73be-9a2d-4441-af53-38419ac7416f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:12.173129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a73be-9a2d-4441-af53-38419ac7416f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:12.173551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a73be-9a2d-4441-af53-38419ac7416f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:12.175361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 687/1371] 10.222.0.26 () {54 vars in 963 bytes} [Mon Jul 11 11:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:12.181220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c156-f81f-4fb7-93f4-86673c51d0bd None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:12.181724 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c156-f81f-4fb7-93f4-86673c51d0bd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:12.182161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c156-f81f-4fb7-93f4-86673c51d0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:12.261687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ed98c156-f81f-4fb7-93f4-86673c51d0bd None None] MFA Rules not processed for user `d7cebda9ae4e4a6b9a9bbface3e7b232`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:12.262881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ed98c156-f81f-4fb7-93f4-86673c51d0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:12.313345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:12.343332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 685/1372] 10.222.0.26 () {58 vars in 1060 bytes} [Mon Jul 11 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:12.350752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a73be-9a2d-4441-af53-38419ac7416f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:12.352269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-220a73be-9a2d-4441-af53-38419ac7416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:12.454859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a73be-9a2d-4441-af53-38419ac7416f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:12.495965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:12.533251 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5f428-85d4-4837-87af-4f9af23e06f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:12.533917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f428-85d4-4837-87af-4f9af23e06f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:12.534201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f428-85d4-4837-87af-4f9af23e06f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:12.534475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5f428-85d4-4837-87af-4f9af23e06f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:12.536170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d2f5f428-85d4-4837-87af-4f9af23e06f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:12.652705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5f428-85d4-4837-87af-4f9af23e06f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:12.654796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5f428-85d4-4837-87af-4f9af23e06f6 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:12.673421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 688/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:12.936237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86b6d-3f7b-4498-a93c-a7492c867453 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:12.936950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86b6d-3f7b-4498-a93c-a7492c867453 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:12.936950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a86b6d-3f7b-4498-a93c-a7492c867453 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:12.943304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 686/1374] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:12.943893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9fc33-8bbe-42c2-b9be-22218d648368 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:12.944412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9fc33-8bbe-42c2-b9be-22218d648368 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:12.944778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9fc33-8bbe-42c2-b9be-22218d648368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:13.025399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-63d9fc33-8bbe-42c2-b9be-22218d648368 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:13.026612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-63d9fc33-8bbe-42c2-b9be-22218d648368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:13.056990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:13.155799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 689/1375] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:18:13.186190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a86b6d-3f7b-4498-a93c-a7492c867453 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:13.187726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-45a86b6d-3f7b-4498-a93c-a7492c867453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:13.350391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a86b6d-3f7b-4498-a93c-a7492c867453 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:13.400595 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:13.462500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42765b6-ae0c-4325-a7aa-2d9724e1db70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:13.463167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42765b6-ae0c-4325-a7aa-2d9724e1db70 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:13.463477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42765b6-ae0c-4325-a7aa-2d9724e1db70 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:13.463748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42765b6-ae0c-4325-a7aa-2d9724e1db70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:13.465371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e42765b6-ae0c-4325-a7aa-2d9724e1db70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:13.551774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42765b6-ae0c-4325-a7aa-2d9724e1db70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:13.553373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42765b6-ae0c-4325-a7aa-2d9724e1db70 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:13.571487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 687/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:15.721561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bfce4-2bd0-44f0-9139-8ac4ee3be7a2 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:15.722089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bfce4-2bd0-44f0-9139-8ac4ee3be7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:15.722206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bfce4-2bd0-44f0-9139-8ac4ee3be7a2 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:15.723531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 690/1377] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:15.739844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4091414-a692-4da9-a2da-f35ca663cc01 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:15.740316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4091414-a692-4da9-a2da-f35ca663cc01 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:15.741862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4091414-a692-4da9-a2da-f35ca663cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:15.824805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c4091414-a692-4da9-a2da-f35ca663cc01 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:15.826018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c4091414-a692-4da9-a2da-f35ca663cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:15.833098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:15.921583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 688/1378] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:18:15.965479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bfce4-2bd0-44f0-9139-8ac4ee3be7a2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:15.966981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1f8bfce4-2bd0-44f0-9139-8ac4ee3be7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:16.120676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bfce4-2bd0-44f0-9139-8ac4ee3be7a2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:16.126769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:16.275174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa4e09-f9cd-4517-9003-c0c1a1d882ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:16.275174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa4e09-f9cd-4517-9003-c0c1a1d882ea service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:16.275174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa4e09-f9cd-4517-9003-c0c1a1d882ea service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:16.275174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa4e09-f9cd-4517-9003-c0c1a1d882ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:16.277071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0dfa4e09-f9cd-4517-9003-c0c1a1d882ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:16.384648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa4e09-f9cd-4517-9003-c0c1a1d882ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:16.386304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa4e09-f9cd-4517-9003-c0c1a1d882ea service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:16.405749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 691/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:18.615578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a130660-7200-4f84-af98-f4b2ecdce815 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:18.616376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a130660-7200-4f84-af98-f4b2ecdce815 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:18.616376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a130660-7200-4f84-af98-f4b2ecdce815 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:18.621407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 689/1380] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:18.623009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8fcb-c2b1-4b05-b51f-c1b98971fa27 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:18.623511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8fcb-c2b1-4b05-b51f-c1b98971fa27 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:18.624167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf8fcb-c2b1-4b05-b51f-c1b98971fa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:18.635488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:18.694141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-23bf8fcb-c2b1-4b05-b51f-c1b98971fa27 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:18.695394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-23bf8fcb-c2b1-4b05-b51f-c1b98971fa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:18.799366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 692/1381] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:18.842857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a130660-7200-4f84-af98-f4b2ecdce815 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:18.844374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7a130660-7200-4f84-af98-f4b2ecdce815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:18.982811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:18.993336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a130660-7200-4f84-af98-f4b2ecdce815 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:19.118208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0630a10-eeff-4b1a-acf1-433cf6a9b11c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:19.118856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0630a10-eeff-4b1a-acf1-433cf6a9b11c service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:19.119137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0630a10-eeff-4b1a-acf1-433cf6a9b11c service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:19.119456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0630a10-eeff-4b1a-acf1-433cf6a9b11c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:19.121085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f0630a10-eeff-4b1a-acf1-433cf6a9b11c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:19.220204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0630a10-eeff-4b1a-acf1-433cf6a9b11c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:19.221880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0630a10-eeff-4b1a-acf1-433cf6a9b11c service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:19.235525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:19.246137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 690/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:21.500775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd1a7-4a1a-4c01-a21e-bfe30196c9c6 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:21.501343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd1a7-4a1a-4c01-a21e-bfe30196c9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:21.501343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fd1a7-4a1a-4c01-a21e-bfe30196c9c6 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:21.502719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 693/1383] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:21.509201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980affa-8a4d-4360-ab01-78052f61fe1c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:21.509763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980affa-8a4d-4360-ab01-78052f61fe1c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:21.509763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980affa-8a4d-4360-ab01-78052f61fe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:21.579178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1980affa-8a4d-4360-ab01-78052f61fe1c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:21.580413 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1980affa-8a4d-4360-ab01-78052f61fe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:21.691284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 691/1384] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:21.737654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fd1a7-4a1a-4c01-a21e-bfe30196c9c6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:21.739209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-685fd1a7-4a1a-4c01-a21e-bfe30196c9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:21.837325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:21.899604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fd1a7-4a1a-4c01-a21e-bfe30196c9c6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:22.015766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41099313-b9aa-42d0-a07f-e8c0827505b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:22.016415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099313-b9aa-42d0-a07f-e8c0827505b0 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:22.016696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099313-b9aa-42d0-a07f-e8c0827505b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:22.016972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41099313-b9aa-42d0-a07f-e8c0827505b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:22.018599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-41099313-b9aa-42d0-a07f-e8c0827505b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:22.085908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:22.127184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41099313-b9aa-42d0-a07f-e8c0827505b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:22.128724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41099313-b9aa-42d0-a07f-e8c0827505b0 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:22.151042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 694/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:24.198678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd77de5-824a-411c-b09e-2f770976eddd None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:24.199492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd77de5-824a-411c-b09e-2f770976eddd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:24.199492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd77de5-824a-411c-b09e-2f770976eddd None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:24.201220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 692/1386] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:24.206338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0d8d-acf0-4cd5-96b1-42a2c5a36b87 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:24.206845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0d8d-acf0-4cd5-96b1-42a2c5a36b87 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:24.207018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0d8d-acf0-4cd5-96b1-42a2c5a36b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:24.278062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1d4d0d8d-acf0-4cd5-96b1-42a2c5a36b87 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:24.279876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1d4d0d8d-acf0-4cd5-96b1-42a2c5a36b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:24.377226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 695/1387] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:24.421243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd77de5-824a-411c-b09e-2f770976eddd None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:24.423118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2cd77de5-824a-411c-b09e-2f770976eddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:24.515808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:24.589062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd77de5-824a-411c-b09e-2f770976eddd None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:24.704186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2af7ca-b350-40a7-a100-4e18032ddbe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:24.704996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af7ca-b350-40a7-a100-4e18032ddbe0 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:24.705347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af7ca-b350-40a7-a100-4e18032ddbe0 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:24.705717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2af7ca-b350-40a7-a100-4e18032ddbe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:24.707935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ab2af7ca-b350-40a7-a100-4e18032ddbe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:24.755661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:24.815471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2af7ca-b350-40a7-a100-4e18032ddbe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:24.817570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2af7ca-b350-40a7-a100-4e18032ddbe0 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:24.849074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 693/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:27.208309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dade6-7eef-491b-a7ec-07e24d77ad04 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:27.208866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dade6-7eef-491b-a7ec-07e24d77ad04 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:27.208949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dade6-7eef-491b-a7ec-07e24d77ad04 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:27.210420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 696/1389] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:27.215806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7550-d610-4526-8ed6-286cc2f57983 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:27.216279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7550-d610-4526-8ed6-286cc2f57983 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:27.216937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec7550-d610-4526-8ed6-286cc2f57983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:27.282436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cdec7550-d610-4526-8ed6-286cc2f57983 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:27.283706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cdec7550-d610-4526-8ed6-286cc2f57983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:27.379188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 694/1390] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:27.423360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dade6-7eef-491b-a7ec-07e24d77ad04 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:27.424874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-217dade6-7eef-491b-a7ec-07e24d77ad04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:27.459581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:27.576689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dade6-7eef-491b-a7ec-07e24d77ad04 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:27.678154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:27.690297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1390ce87-d693-4d40-99e6-f83784e61731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:27.691045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ce87-d693-4d40-99e6-f83784e61731 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:27.691391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ce87-d693-4d40-99e6-f83784e61731 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:27.691666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1390ce87-d693-4d40-99e6-f83784e61731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:27.693406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1390ce87-d693-4d40-99e6-f83784e61731 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:27.783638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390ce87-d693-4d40-99e6-f83784e61731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:27.785288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1390ce87-d693-4d40-99e6-f83784e61731 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:27.805377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 697/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:28.070165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e4743-0b46-41b9-9857-1990fb6894cd None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:28.070501 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e4743-0b46-41b9-9857-1990fb6894cd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:28.070675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e4743-0b46-41b9-9857-1990fb6894cd None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:28.072532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 695/1392] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:28.078733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c324d-4dae-422b-b377-7f401c203a1b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:28.079194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c324d-4dae-422b-b377-7f401c203a1b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:28.079470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c324d-4dae-422b-b377-7f401c203a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:28.167884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a91c324d-4dae-422b-b377-7f401c203a1b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:28.169186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a91c324d-4dae-422b-b377-7f401c203a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:28.231705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:28.270322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 698/1393] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:28.328478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13519da8-7532-45f4-801f-332ba16d6b61 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:28.328963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13519da8-7532-45f4-801f-332ba16d6b61 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:28.329175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13519da8-7532-45f4-801f-332ba16d6b61 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:28.330616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 696/1394] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jul 11 11:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:28.335989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4112e4-756a-4c65-8d21-aac655e15e92 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:28.336352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4112e4-756a-4c65-8d21-aac655e15e92 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:28.336735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4112e4-756a-4c65-8d21-aac655e15e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:28.415301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ef4112e4-756a-4c65-8d21-aac655e15e92 None None] MFA Rules not processed for user `e939bc73c40140cb89517e918722d154`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:28.416518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ef4112e4-756a-4c65-8d21-aac655e15e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:28.492396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 699/1395] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jul 11 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:28.499879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13519da8-7532-45f4-801f-332ba16d6b61 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:28.501376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-13519da8-7532-45f4-801f-332ba16d6b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:28.514708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:28.606483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13519da8-7532-45f4-801f-332ba16d6b61 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:28.681034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b3c16-ce86-4b5b-93c0-7c1ff826e692 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e939bc73c40140cb89517e918722d154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:28.681702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b3c16-ce86-4b5b-93c0-7c1ff826e692 service glance] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:28.681993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b3c16-ce86-4b5b-93c0-7c1ff826e692 service glance] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:28.682277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b3c16-ce86-4b5b-93c0-7c1ff826e692 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:28.684038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e70b3c16-ce86-4b5b-93c0-7c1ff826e692 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:28.752042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:28.792760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b3c16-ce86-4b5b-93c0-7c1ff826e692 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:28.794265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b3c16-ce86-4b5b-93c0-7c1ff826e692 service glance] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:28.813772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 697/1396] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:29.687534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbde0d-57f8-4a6c-bc8c-76e634468de1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:29.688053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbde0d-57f8-4a6c-bc8c-76e634468de1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:29.688163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbde0d-57f8-4a6c-bc8c-76e634468de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:29.759927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-72dbde0d-57f8-4a6c-bc8c-76e634468de1 None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:29.761154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-72dbde0d-57f8-4a6c-bc8c-76e634468de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:29.770476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:29.849090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 700/1397] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:29.896765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a261f-502e-4929-b785-ef227b14b57a None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:29.898981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a261f-502e-4929-b785-ef227b14b57a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:29.898981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a261f-502e-4929-b785-ef227b14b57a None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:29.898981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 698/1398] 10.222.0.26 () {54 vars in 942 bytes} [Mon Jul 11 11:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:29.905212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeb55f-964d-4591-997d-0cb4dd43e86c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:29.905506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeb55f-964d-4591-997d-0cb4dd43e86c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:29.905790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeb55f-964d-4591-997d-0cb4dd43e86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:29.981098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7663-d682-4727-b603-e233d00ce27c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:29.981293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7663-d682-4727-b603-e233d00ce27c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:29.981404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-92aeb55f-964d-4591-997d-0cb4dd43e86c None None] MFA Rules not processed for user `7556ce4bf09d4113aec2037dd438cecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:29.981566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f7663-d682-4727-b603-e233d00ce27c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:29.982622 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-92aeb55f-964d-4591-997d-0cb4dd43e86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:29.982964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 699/1399] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:18:29.988204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8157b-2607-4b4d-9c35-b0fa4892c2f8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:29.988489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8157b-2607-4b4d-9c35-b0fa4892c2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:29.988761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8157b-2607-4b4d-9c35-b0fa4892c2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:30.056311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-11d8157b-2607-4b4d-9c35-b0fa4892c2f8 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:30.057540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-11d8157b-2607-4b4d-9c35-b0fa4892c2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:30.078870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:30.100942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 701/1400] 10.222.0.26 () {58 vars in 1039 bytes} [Mon Jul 11 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:30.111382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aeb55f-964d-4591-997d-0cb4dd43e86c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:30.123309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aeb55f-964d-4591-997d-0cb4dd43e86c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:30.162308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 700/1401] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:18:30.210148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8157b-2607-4b4d-9c35-b0fa4892c2f8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:30.211479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-11d8157b-2607-4b4d-9c35-b0fa4892c2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:30.246522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e793cf-8cac-44f9-8e2b-021f9a580dc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:30.247093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e793cf-8cac-44f9-8e2b-021f9a580dc5 service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:30.247490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e793cf-8cac-44f9-8e2b-021f9a580dc5 service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:30.248277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e793cf-8cac-44f9-8e2b-021f9a580dc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:30.248384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:30.249987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f2e793cf-8cac-44f9-8e2b-021f9a580dc5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:30.341705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e793cf-8cac-44f9-8e2b-021f9a580dc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:30.343381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e793cf-8cac-44f9-8e2b-021f9a580dc5 service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:30.348427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8157b-2607-4b4d-9c35-b0fa4892c2f8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:30.352323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 702/1402] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 11:18:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:18:30.465965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df18bf1-66be-479b-a3e6-01478b4ab1d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:30.467143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df18bf1-66be-479b-a3e6-01478b4ab1d2 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:30.467456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df18bf1-66be-479b-a3e6-01478b4ab1d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:30.467737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df18bf1-66be-479b-a3e6-01478b4ab1d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:30.469404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7df18bf1-66be-479b-a3e6-01478b4ab1d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:30.487492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:30.558073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df18bf1-66be-479b-a3e6-01478b4ab1d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:30.559601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df18bf1-66be-479b-a3e6-01478b4ab1d2 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:30.577259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 701/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:18:32.782070 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b933-a458-4e40-9638-ce5e5d7c4324 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:32.782636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b933-a458-4e40-9638-ce5e5d7c4324 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:32.782636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b933-a458-4e40-9638-ce5e5d7c4324 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:32.785294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 703/1404] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:32.789699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd7f76-d8ed-4f71-be37-adfd728a2678 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:32.790111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd7f76-d8ed-4f71-be37-adfd728a2678 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:32.790274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd7f76-d8ed-4f71-be37-adfd728a2678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:32.861761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3acd7f76-d8ed-4f71-be37-adfd728a2678 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:32.863097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3acd7f76-d8ed-4f71-be37-adfd728a2678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:32.972495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:32.987349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 702/1405] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:33.024877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3b933-a458-4e40-9638-ce5e5d7c4324 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:33.026431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-33b3b933-a458-4e40-9638-ce5e5d7c4324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:33.036635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:33.206635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3b933-a458-4e40-9638-ce5e5d7c4324 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:33.307294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:33.345700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b77730-6f89-4055-af28-154c76d43fcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:33.346448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b77730-6f89-4055-af28-154c76d43fcc service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:33.346805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b77730-6f89-4055-af28-154c76d43fcc service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:33.349108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b77730-6f89-4055-af28-154c76d43fcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:33.349108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-07b77730-6f89-4055-af28-154c76d43fcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:33.478396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b77730-6f89-4055-af28-154c76d43fcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:33.480891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b77730-6f89-4055-af28-154c76d43fcc service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:33.500955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 704/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:36.325183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343548b-3d3c-4f9c-be9a-9be7f0f30021 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:36.325719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343548b-3d3c-4f9c-be9a-9be7f0f30021 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:36.325719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343548b-3d3c-4f9c-be9a-9be7f0f30021 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:36.333016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7083f-58d4-4918-836a-29e1b2266809 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:36.333545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7083f-58d4-4918-836a-29e1b2266809 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:36.333545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7083f-58d4-4918-836a-29e1b2266809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:36.335391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 703/1407] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:18:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:36.401916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e2c7083f-58d4-4918-836a-29e1b2266809 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:36.403118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e2c7083f-58d4-4918-836a-29e1b2266809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:36.453047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:36.509994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 705/1408] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:18:36.554455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343548b-3d3c-4f9c-be9a-9be7f0f30021 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:36.556317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a343548b-3d3c-4f9c-be9a-9be7f0f30021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:36.720424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343548b-3d3c-4f9c-be9a-9be7f0f30021 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:36.793203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:36.849048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ee68f-b1d6-4227-b7ef-0c6e117caf2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:36.849864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ee68f-b1d6-4227-b7ef-0c6e117caf2e service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:36.850206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ee68f-b1d6-4227-b7ef-0c6e117caf2e service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:36.850557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ee68f-b1d6-4227-b7ef-0c6e117caf2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:36.852290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-894ee68f-b1d6-4227-b7ef-0c6e117caf2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:36.960001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ee68f-b1d6-4227-b7ef-0c6e117caf2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:36.961807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ee68f-b1d6-4227-b7ef-0c6e117caf2e service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:36.982281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 704/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:46.507913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=23 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:46.979657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=64 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:54.386121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92120fee-f101-4481-8997-2eae79771cac None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:54.386643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92120fee-f101-4481-8997-2eae79771cac None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:54.386643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92120fee-f101-4481-8997-2eae79771cac None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:54.388004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 706/1410] 10.222.0.26 () {54 vars in 918 bytes} [Mon Jul 11 11:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:54.393599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24e834-6b7e-406b-9bdc-c66a61774887 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:54.393997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24e834-6b7e-406b-9bdc-c66a61774887 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:54.394196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24e834-6b7e-406b-9bdc-c66a61774887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:54.467698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7d24e834-6b7e-406b-9bdc-c66a61774887 None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:54.468919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7d24e834-6b7e-406b-9bdc-c66a61774887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:54.571809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 705/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Jul 11 11:18:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:54.586896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd767b-c49a-449a-b008-4781b254adc0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:54.587278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd767b-c49a-449a-b008-4781b254adc0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:54.587576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd767b-c49a-449a-b008-4781b254adc0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:54.588973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 707/1412] 10.222.0.26 () {54 vars in 918 bytes} [Mon Jul 11 11:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:54.593785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c94aab-7e88-49f3-8598-66276e607b98 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:54.594079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c94aab-7e88-49f3-8598-66276e607b98 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:54.594359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c94aab-7e88-49f3-8598-66276e607b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:54.665130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:54.668186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d2c94aab-7e88-49f3-8598-66276e607b98 None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:54.669400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d2c94aab-7e88-49f3-8598-66276e607b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:54.783541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 706/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Mon Jul 11 11:18:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:18:57.311632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120a76e-3032-48a0-88bf-03cccc09ae41 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:57.312157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120a76e-3032-48a0-88bf-03cccc09ae41 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:57.312280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120a76e-3032-48a0-88bf-03cccc09ae41 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:57.313568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:57.318986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69683f-4615-4fae-b7f5-342a538bcdb4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:57.319407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69683f-4615-4fae-b7f5-342a538bcdb4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:57.319607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69683f-4615-4fae-b7f5-342a538bcdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:57.382398 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ed69683f-4615-4fae-b7f5-342a538bcdb4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:57.383562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ed69683f-4615-4fae-b7f5-342a538bcdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:57.437721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:57.478995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:57.521297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120a76e-3032-48a0-88bf-03cccc09ae41 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:57.522806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a120a76e-3032-48a0-88bf-03cccc09ae41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:57.671160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120a76e-3032-48a0-88bf-03cccc09ae41 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:57.743330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:57.783280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ff2f5-8c4c-42c8-aea5-10c96543acc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:57.783939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ff2f5-8c4c-42c8-aea5-10c96543acc1 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:57.784225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ff2f5-8c4c-42c8-aea5-10c96543acc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:57.784505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ff2f5-8c4c-42c8-aea5-10c96543acc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:57.786187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c44ff2f5-8c4c-42c8-aea5-10c96543acc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:57.882278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ff2f5-8c4c-42c8-aea5-10c96543acc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:57.883939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ff2f5-8c4c-42c8-aea5-10c96543acc1 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:57.901734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 709/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:18:58.800739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efbb27-9be4-414a-9026-621cc03d5994 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:58.801376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efbb27-9be4-414a-9026-621cc03d5994 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:58.801376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efbb27-9be4-414a-9026-621cc03d5994 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:58.802710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:18:58.807900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c43f-8d65-48f2-90cd-376a54395a4c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:58.808314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c43f-8d65-48f2-90cd-376a54395a4c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:58.808460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9c43f-8d65-48f2-90cd-376a54395a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:58.881059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-add9c43f-8d65-48f2-90cd-376a54395a4c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:18:58.882255 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-add9c43f-8d65-48f2-90cd-376a54395a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:58.908459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:58.976156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:18:59.140321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efbb27-9be4-414a-9026-621cc03d5994 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:18:59.141817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-30efbb27-9be4-414a-9026-621cc03d5994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:59.283018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efbb27-9be4-414a-9026-621cc03d5994 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:18:59.322413 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:18:59.393856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbdee3-a122-497e-ab99-199a1d6a18df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:18:59.394427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbdee3-a122-497e-ab99-199a1d6a18df service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:18:59.394715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbdee3-a122-497e-ab99-199a1d6a18df service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:18:59.394991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbdee3-a122-497e-ab99-199a1d6a18df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:18:59.396653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cabbdee3-a122-497e-ab99-199a1d6a18df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:18:59.483350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbdee3-a122-497e-ab99-199a1d6a18df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:18:59.484956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbdee3-a122-497e-ab99-199a1d6a18df service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:18:59.504849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 709/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:00.376294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2ef7-09cc-48a2-a78e-56b28db0b493 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:00.376858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2ef7-09cc-48a2-a78e-56b28db0b493 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:00.376858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b2ef7-09cc-48a2-a78e-56b28db0b493 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:00.378130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:00.382907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e53b-eff7-4326-97d0-46e154a8c8f1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:00.383355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e53b-eff7-4326-97d0-46e154a8c8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:00.383495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59e53b-eff7-4326-97d0-46e154a8c8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:00.436036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:00.456887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0e59e53b-eff7-4326-97d0-46e154a8c8f1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:00.458098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0e59e53b-eff7-4326-97d0-46e154a8c8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:00.551729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:00.703499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b2ef7-09cc-48a2-a78e-56b28db0b493 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:00.705049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1c7b2ef7-09cc-48a2-a78e-56b28db0b493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:00.849503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b2ef7-09cc-48a2-a78e-56b28db0b493 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:00.861956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:00.961778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c7bbc-f04a-400f-a53d-57164e5a7265 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:00.962341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7bbc-f04a-400f-a53d-57164e5a7265 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:00.962625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7bbc-f04a-400f-a53d-57164e5a7265 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:00.962900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7bbc-f04a-400f-a53d-57164e5a7265 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:00.964596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9e9c7bbc-f04a-400f-a53d-57164e5a7265 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:01.051600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c7bbc-f04a-400f-a53d-57164e5a7265 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:01.053194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c7bbc-f04a-400f-a53d-57164e5a7265 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:01.071475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 712/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:01.921168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85b8d1-1ea8-4b0a-89ff-540e9d06aceb None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:01.921767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85b8d1-1ea8-4b0a-89ff-540e9d06aceb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:01.921767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85b8d1-1ea8-4b0a-89ff-540e9d06aceb None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:01.923064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:01.927939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b28e83-5015-49de-9923-ea0131a92560 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:01.928343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b28e83-5015-49de-9923-ea0131a92560 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:01.928499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b28e83-5015-49de-9923-ea0131a92560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:01.947429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:02.003649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f3b28e83-5015-49de-9923-ea0131a92560 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:02.004847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f3b28e83-5015-49de-9923-ea0131a92560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:02.098909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:02.246349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85b8d1-1ea8-4b0a-89ff-540e9d06aceb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:02.248358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1a85b8d1-1ea8-4b0a-89ff-540e9d06aceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:02.353762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:02.388060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85b8d1-1ea8-4b0a-89ff-540e9d06aceb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:02.499493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75ad63-b4f1-4f59-960e-4844cb220252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:02.500054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75ad63-b4f1-4f59-960e-4844cb220252 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:02.500341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75ad63-b4f1-4f59-960e-4844cb220252 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:02.500618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75ad63-b4f1-4f59-960e-4844cb220252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:02.502247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3e75ad63-b4f1-4f59-960e-4844cb220252 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:02.575603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:02.589302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75ad63-b4f1-4f59-960e-4844cb220252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:02.590888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75ad63-b4f1-4f59-960e-4844cb220252 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:02.609526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 712/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:03.472784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be7d-732c-4fcf-b28c-6475036254c5 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:03.473305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be7d-732c-4fcf-b28c-6475036254c5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:03.473305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057be7d-732c-4fcf-b28c-6475036254c5 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:03.474656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:03.479631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b739-7c78-4af3-a01f-9af2c3aa4181 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:03.480080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b739-7c78-4af3-a01f-9af2c3aa4181 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:03.480180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b739-7c78-4af3-a01f-9af2c3aa4181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:03.553461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7b21b739-7c78-4af3-a01f-9af2c3aa4181 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:03.554947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7b21b739-7c78-4af3-a01f-9af2c3aa4181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:03.649016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:19:03.799561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057be7d-732c-4fcf-b28c-6475036254c5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:03.801018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7057be7d-732c-4fcf-b28c-6475036254c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:03.888084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:03.949955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057be7d-732c-4fcf-b28c-6475036254c5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:04.065045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665939b8-0672-439a-b516-41acd80af328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:04.065620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665939b8-0672-439a-b516-41acd80af328 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:04.065905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665939b8-0672-439a-b516-41acd80af328 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:04.066182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665939b8-0672-439a-b516-41acd80af328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:04.067897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-665939b8-0672-439a-b516-41acd80af328 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:04.117022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:04.157931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665939b8-0672-439a-b516-41acd80af328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:04.159570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665939b8-0672-439a-b516-41acd80af328 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:04.177877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 715/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:05.047629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc948ce-7234-425b-9348-1242902cebc4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:05.048218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc948ce-7234-425b-9348-1242902cebc4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:05.048218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc948ce-7234-425b-9348-1242902cebc4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:05.049625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:05.054914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596271a2-92ef-403f-8ddf-cdf04739df04 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:05.055402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596271a2-92ef-403f-8ddf-cdf04739df04 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:05.055620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596271a2-92ef-403f-8ddf-cdf04739df04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:05.128708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-596271a2-92ef-403f-8ddf-cdf04739df04 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:05.129921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-596271a2-92ef-403f-8ddf-cdf04739df04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:05.224960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:05.373993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc948ce-7234-425b-9348-1242902cebc4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:05.375580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5bc948ce-7234-425b-9348-1242902cebc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:05.425047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:05.521197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc948ce-7234-425b-9348-1242902cebc4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:05.633403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f277b-ea63-4338-98c2-4a75e4017d0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:05.633864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f277b-ea63-4338-98c2-4a75e4017d0e service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:05.634145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f277b-ea63-4338-98c2-4a75e4017d0e service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:05.634419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f277b-ea63-4338-98c2-4a75e4017d0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:05.636155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-784f277b-ea63-4338-98c2-4a75e4017d0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:05.643561 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:05.724767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f277b-ea63-4338-98c2-4a75e4017d0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:05.726341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f277b-ea63-4338-98c2-4a75e4017d0e service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:05.744102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 715/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:06.594743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7a118-35d6-4510-9be8-d7e77613efde None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:06.595286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7a118-35d6-4510-9be8-d7e77613efde None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:06.595286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7a118-35d6-4510-9be8-d7e77613efde None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:06.596635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:06.601695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867f1cd-c441-4d31-b221-92aeffab6a53 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:06.602131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867f1cd-c441-4d31-b221-92aeffab6a53 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:06.602268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867f1cd-c441-4d31-b221-92aeffab6a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:06.674666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b867f1cd-c441-4d31-b221-92aeffab6a53 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:06.675913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b867f1cd-c441-4d31-b221-92aeffab6a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:06.742403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:06.772450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:06.922329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7a118-35d6-4510-9be8-d7e77613efde None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:06.923910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-92f7a118-35d6-4510-9be8-d7e77613efde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:06.941908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:07.069027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7a118-35d6-4510-9be8-d7e77613efde None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:07.160719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:07.191367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bf585-dca9-4a1c-b088-4940f40d711c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:07.192019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf585-dca9-4a1c-b088-4940f40d711c service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:07.192301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf585-dca9-4a1c-b088-4940f40d711c service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:07.192574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bf585-dca9-4a1c-b088-4940f40d711c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:07.194220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b07bf585-dca9-4a1c-b088-4940f40d711c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:07.281085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bf585-dca9-4a1c-b088-4940f40d711c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:07.282683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bf585-dca9-4a1c-b088-4940f40d711c service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:07.300713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 718/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:08.138712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90281a9e-869c-414e-b13a-306fac62c673 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:08.139286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90281a9e-869c-414e-b13a-306fac62c673 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:08.139390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90281a9e-869c-414e-b13a-306fac62c673 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:08.140765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:08.145660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366db58f-7a7a-429d-82cc-4d9f3acc3078 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:08.146083 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366db58f-7a7a-429d-82cc-4d9f3acc3078 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:08.146205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366db58f-7a7a-429d-82cc-4d9f3acc3078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:08.219617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-366db58f-7a7a-429d-82cc-4d9f3acc3078 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:08.220945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-366db58f-7a7a-429d-82cc-4d9f3acc3078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:08.260976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:08.315448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:08.467445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90281a9e-869c-414e-b13a-306fac62c673 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:08.469034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-90281a9e-869c-414e-b13a-306fac62c673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:08.609879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90281a9e-869c-414e-b13a-306fac62c673 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:08.662631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:08.720684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d758a12-a96d-4e78-a257-92859253302f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:08.721363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d758a12-a96d-4e78-a257-92859253302f service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:08.721650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d758a12-a96d-4e78-a257-92859253302f service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:08.721934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d758a12-a96d-4e78-a257-92859253302f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:08.723632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8d758a12-a96d-4e78-a257-92859253302f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:08.811707 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d758a12-a96d-4e78-a257-92859253302f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:08.813293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d758a12-a96d-4e78-a257-92859253302f service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:08.831466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 718/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:09.684311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069a550-64da-4d2b-9d12-dd1e0eba551d None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:09.685364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069a550-64da-4d2b-9d12-dd1e0eba551d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:09.685364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069a550-64da-4d2b-9d12-dd1e0eba551d None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:09.686753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:09.691795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd81a04-f07f-431d-b850-e7470e5dd159 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:09.692252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd81a04-f07f-431d-b850-e7470e5dd159 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:09.692379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd81a04-f07f-431d-b850-e7470e5dd159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:09.766270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bfd81a04-f07f-431d-b850-e7470e5dd159 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:09.767525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bfd81a04-f07f-431d-b850-e7470e5dd159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:09.769514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:09.862262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:10.013876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069a550-64da-4d2b-9d12-dd1e0eba551d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:10.015435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8069a550-64da-4d2b-9d12-dd1e0eba551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:10.159093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069a550-64da-4d2b-9d12-dd1e0eba551d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:10.186763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:10.270737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048e030-9793-4401-8251-cd0eff327c81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:10.271400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e030-9793-4401-8251-cd0eff327c81 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:10.271684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e030-9793-4401-8251-cd0eff327c81 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:10.271955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048e030-9793-4401-8251-cd0eff327c81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:10.273584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1048e030-9793-4401-8251-cd0eff327c81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:10.360941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048e030-9793-4401-8251-cd0eff327c81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:10.362523 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048e030-9793-4401-8251-cd0eff327c81 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:10.379951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 721/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:12.318055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db878598-d60e-4bd1-b3a2-14af22b169d4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:12.318623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db878598-d60e-4bd1-b3a2-14af22b169d4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:12.318623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db878598-d60e-4bd1-b3a2-14af22b169d4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:12.320029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:12.325213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db347c37-5810-4547-940b-6515c6edb132 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:12.326160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db347c37-5810-4547-940b-6515c6edb132 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:12.326258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db347c37-5810-4547-940b-6515c6edb132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:12.364592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:12.388961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-db347c37-5810-4547-940b-6515c6edb132 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:12.390161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-db347c37-5810-4547-940b-6515c6edb132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:12.484496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:12.522858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db878598-d60e-4bd1-b3a2-14af22b169d4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:12.524406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-db878598-d60e-4bd1-b3a2-14af22b169d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:12.624867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db878598-d60e-4bd1-b3a2-14af22b169d4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:12.655829 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:12.696206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41646d9-c594-4b5d-b621-35f2ec059884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:12.696853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41646d9-c594-4b5d-b621-35f2ec059884 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:12.697144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41646d9-c594-4b5d-b621-35f2ec059884 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:12.697423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41646d9-c594-4b5d-b621-35f2ec059884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:12.699080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a41646d9-c594-4b5d-b621-35f2ec059884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:12.804220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41646d9-c594-4b5d-b621-35f2ec059884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:12.805905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41646d9-c594-4b5d-b621-35f2ec059884 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:12.824057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 721/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:15.157993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc4cdf-d3db-4e7e-825f-2c82d45f19db None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:15.158570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc4cdf-d3db-4e7e-825f-2c82d45f19db None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:15.158570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc4cdf-d3db-4e7e-825f-2c82d45f19db None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:15.159962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:15.165166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b23c6-703e-4eec-84e5-f2ba5a4d32a4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:15.165566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b23c6-703e-4eec-84e5-f2ba5a4d32a4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:15.165727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b23c6-703e-4eec-84e5-f2ba5a4d32a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:15.235369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ef6b23c6-703e-4eec-84e5-f2ba5a4d32a4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:15.236591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ef6b23c6-703e-4eec-84e5-f2ba5a4d32a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:15.249754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:15.331791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:15.370092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc4cdf-d3db-4e7e-825f-2c82d45f19db None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:15.371703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-63cc4cdf-d3db-4e7e-825f-2c82d45f19db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:15.475179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc4cdf-d3db-4e7e-825f-2c82d45f19db None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:15.486981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:15.547807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b013284-009f-4ccc-a25d-7f3d76beed16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:15.548460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b013284-009f-4ccc-a25d-7f3d76beed16 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:15.548746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b013284-009f-4ccc-a25d-7f3d76beed16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:15.549024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b013284-009f-4ccc-a25d-7f3d76beed16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:15.550668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4b013284-009f-4ccc-a25d-7f3d76beed16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:15.658766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b013284-009f-4ccc-a25d-7f3d76beed16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:15.660146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b013284-009f-4ccc-a25d-7f3d76beed16 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:15.678060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 724/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:18.071525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4b42-1382-4554-b15b-af2858c23cbf None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:18.072060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4b42-1382-4554-b15b-af2858c23cbf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:18.072060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec4b42-1382-4554-b15b-af2858c23cbf None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:18.073452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:18.078771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca7a2c-2fdc-410e-a2bd-bfb869fa4a00 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:18.079163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca7a2c-2fdc-410e-a2bd-bfb869fa4a00 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:18.079367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca7a2c-2fdc-410e-a2bd-bfb869fa4a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:18.125175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:18.142241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-10ca7a2c-2fdc-410e-a2bd-bfb869fa4a00 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:18.143468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-10ca7a2c-2fdc-410e-a2bd-bfb869fa4a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:18.236939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:18.275366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec4b42-1382-4554-b15b-af2858c23cbf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:18.276885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3fec4b42-1382-4554-b15b-af2858c23cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:18.377682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec4b42-1382-4554-b15b-af2858c23cbf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:18.416501 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:18.449302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17041c3a-3808-4020-908b-d5b963770620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:18.449942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17041c3a-3808-4020-908b-d5b963770620 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:18.450759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17041c3a-3808-4020-908b-d5b963770620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:18.451035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17041c3a-3808-4020-908b-d5b963770620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:18.452677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-17041c3a-3808-4020-908b-d5b963770620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:18.557677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17041c3a-3808-4020-908b-d5b963770620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:18.559118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17041c3a-3808-4020-908b-d5b963770620 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:18.578099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 724/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:20.701826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f2f1-af92-4ee8-95fa-2305d9ac832f None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:20.702419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f2f1-af92-4ee8-95fa-2305d9ac832f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:20.702419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652f2f1-af92-4ee8-95fa-2305d9ac832f None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:20.703753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:20.709084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3024-4d44-4a86-b4d7-81cb7e8d5593 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:20.709544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3024-4d44-4a86-b4d7-81cb7e8d5593 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:20.709657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f3024-4d44-4a86-b4d7-81cb7e8d5593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:20.774459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ce3f3024-4d44-4a86-b4d7-81cb7e8d5593 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:20.775726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ce3f3024-4d44-4a86-b4d7-81cb7e8d5593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:20.796879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:20.871047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:20.909284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652f2f1-af92-4ee8-95fa-2305d9ac832f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:20.910812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7652f2f1-af92-4ee8-95fa-2305d9ac832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:21.024658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652f2f1-af92-4ee8-95fa-2305d9ac832f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:21.051351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:21.109617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effceb7-1d02-433c-bb21-6bd8168ff532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:21.110234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effceb7-1d02-433c-bb21-6bd8168ff532 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:21.110511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effceb7-1d02-433c-bb21-6bd8168ff532 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:21.110784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effceb7-1d02-433c-bb21-6bd8168ff532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:21.112450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5effceb7-1d02-433c-bb21-6bd8168ff532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:21.219111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effceb7-1d02-433c-bb21-6bd8168ff532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:21.220938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effceb7-1d02-433c-bb21-6bd8168ff532 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:21.240529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 727/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:19:23.347575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1af3d-2fcf-463e-8a59-4a9cb917b8ff None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:23.348113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1af3d-2fcf-463e-8a59-4a9cb917b8ff None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:23.348113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1af3d-2fcf-463e-8a59-4a9cb917b8ff None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:23.349419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:23.354706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f1de8-5e4f-47eb-9261-e4a9a8932108 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:23.357638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f1de8-5e4f-47eb-9261-e4a9a8932108 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:23.357638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f1de8-5e4f-47eb-9261-e4a9a8932108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:23.417596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:23.420791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e42f1de8-5e4f-47eb-9261-e4a9a8932108 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:23.422005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e42f1de8-5e4f-47eb-9261-e4a9a8932108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:23.517594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:23.555180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1af3d-2fcf-463e-8a59-4a9cb917b8ff None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:23.556705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6fe1af3d-2fcf-463e-8a59-4a9cb917b8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:23.658203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1af3d-2fcf-463e-8a59-4a9cb917b8ff None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:23.705910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:23.735188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936d13b-8d93-43e2-9d4f-9a6190e61691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:23.735864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d13b-8d93-43e2-9d4f-9a6190e61691 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:23.736152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d13b-8d93-43e2-9d4f-9a6190e61691 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:23.736437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d13b-8d93-43e2-9d4f-9a6190e61691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:23.738732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f936d13b-8d93-43e2-9d4f-9a6190e61691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:23.842764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936d13b-8d93-43e2-9d4f-9a6190e61691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:23.844199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936d13b-8d93-43e2-9d4f-9a6190e61691 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:23.862027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 727/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:25.945779 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80010e-f0b4-49d5-83c4-9e98c88f4fc1 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:25.946333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80010e-f0b4-49d5-83c4-9e98c88f4fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:25.946461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80010e-f0b4-49d5-83c4-9e98c88f4fc1 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:25.947774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:25.953473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440ad63-a7e5-43a8-96aa-7e3f03acdd64 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:25.953920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440ad63-a7e5-43a8-96aa-7e3f03acdd64 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:25.954030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440ad63-a7e5-43a8-96aa-7e3f03acdd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:26.018627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d440ad63-a7e5-43a8-96aa-7e3f03acdd64 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:26.019868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d440ad63-a7e5-43a8-96aa-7e3f03acdd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:26.048851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:26.116964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:26.155317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80010e-f0b4-49d5-83c4-9e98c88f4fc1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:26.156835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ce80010e-f0b4-49d5-83c4-9e98c88f4fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:26.258765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80010e-f0b4-49d5-83c4-9e98c88f4fc1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:26.286918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:26.331051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77037b8-36c4-4660-8ca2-dff7d7ca6748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:26.331719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77037b8-36c4-4660-8ca2-dff7d7ca6748 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:26.332007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77037b8-36c4-4660-8ca2-dff7d7ca6748 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:26.332284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77037b8-36c4-4660-8ca2-dff7d7ca6748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:26.333917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d77037b8-36c4-4660-8ca2-dff7d7ca6748 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:26.440325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77037b8-36c4-4660-8ca2-dff7d7ca6748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:26.441763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77037b8-36c4-4660-8ca2-dff7d7ca6748 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:26.460107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 730/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:19:28.551135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12426ab3-c06d-4aa3-9e50-fc92ae436ddd None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:28.551731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12426ab3-c06d-4aa3-9e50-fc92ae436ddd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:28.551731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12426ab3-c06d-4aa3-9e50-fc92ae436ddd None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:28.553074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:28.558576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c0f3c-d990-46cd-b571-ed68c760c544 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:28.559034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c0f3c-d990-46cd-b571-ed68c760c544 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:28.559730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c0f3c-d990-46cd-b571-ed68c760c544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:28.623603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3a4c0f3c-d990-46cd-b571-ed68c760c544 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:28.624810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3a4c0f3c-d990-46cd-b571-ed68c760c544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:28.633148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:28.721138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:28.758511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12426ab3-c06d-4aa3-9e50-fc92ae436ddd None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:28.760049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-12426ab3-c06d-4aa3-9e50-fc92ae436ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:28.860442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12426ab3-c06d-4aa3-9e50-fc92ae436ddd None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:28.914069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:28.933207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dbcb03-cb6c-4945-a077-374388366814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:28.933855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbcb03-cb6c-4945-a077-374388366814 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:28.934283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbcb03-cb6c-4945-a077-374388366814 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:28.934565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dbcb03-cb6c-4945-a077-374388366814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:28.936248 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f6dbcb03-cb6c-4945-a077-374388366814 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:29.041655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dbcb03-cb6c-4945-a077-374388366814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:29.043122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dbcb03-cb6c-4945-a077-374388366814 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:29.062351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 730/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:31.132027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec013c-600b-4dac-86ed-ef343a528190 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:31.132627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec013c-600b-4dac-86ed-ef343a528190 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:31.132627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec013c-600b-4dac-86ed-ef343a528190 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:31.133948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:31.139179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84ccae-b5b3-4e0c-b672-97582d951c17 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:31.139682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84ccae-b5b3-4e0c-b672-97582d951c17 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:31.139797 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84ccae-b5b3-4e0c-b672-97582d951c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:31.202540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-df84ccae-b5b3-4e0c-b672-97582d951c17 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:31.203760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-df84ccae-b5b3-4e0c-b672-97582d951c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:31.242979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:31.302205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:31.339462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec013c-600b-4dac-86ed-ef343a528190 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:31.341620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-81ec013c-600b-4dac-86ed-ef343a528190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:31.447385 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec013c-600b-4dac-86ed-ef343a528190 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:31.484083 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:31.519793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97032d-6764-42fd-8f21-5ba876f7fc26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:31.520439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97032d-6764-42fd-8f21-5ba876f7fc26 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:31.520723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97032d-6764-42fd-8f21-5ba876f7fc26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:31.520998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97032d-6764-42fd-8f21-5ba876f7fc26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:31.522636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7e97032d-6764-42fd-8f21-5ba876f7fc26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:31.629709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97032d-6764-42fd-8f21-5ba876f7fc26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:31.631059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97032d-6764-42fd-8f21-5ba876f7fc26 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:31.648429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 733/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:33.696513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43c90-1f1c-44c8-9ef2-8fd289d9651e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:33.697086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43c90-1f1c-44c8-9ef2-8fd289d9651e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:33.697086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43c90-1f1c-44c8-9ef2-8fd289d9651e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:33.698465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:33.704596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe5916-9e39-476e-9ed5-8f077c144fb9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:33.705050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe5916-9e39-476e-9ed5-8f077c144fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:33.705161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe5916-9e39-476e-9ed5-8f077c144fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:33.767690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-25fe5916-9e39-476e-9ed5-8f077c144fb9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:33.769103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-25fe5916-9e39-476e-9ed5-8f077c144fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:33.786095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:33.861760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:33.898089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac43c90-1f1c-44c8-9ef2-8fd289d9651e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:33.899666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4ac43c90-1f1c-44c8-9ef2-8fd289d9651e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:34.002545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac43c90-1f1c-44c8-9ef2-8fd289d9651e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:34.069087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:34.076233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b15fe4-e656-4f77-9fd7-5c5885eb36a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:34.076887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b15fe4-e656-4f77-9fd7-5c5885eb36a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:34.077167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b15fe4-e656-4f77-9fd7-5c5885eb36a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:34.077440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b15fe4-e656-4f77-9fd7-5c5885eb36a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:34.079126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b7b15fe4-e656-4f77-9fd7-5c5885eb36a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:34.190136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b15fe4-e656-4f77-9fd7-5c5885eb36a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:34.191608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b15fe4-e656-4f77-9fd7-5c5885eb36a1 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:34.209277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 733/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:36.237509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc54c5-45c5-4a4b-8ee3-a7b492e67457 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:36.238081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc54c5-45c5-4a4b-8ee3-a7b492e67457 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:36.238081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc54c5-45c5-4a4b-8ee3-a7b492e67457 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:36.239522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:19:36.244605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50861a77-4462-4c07-96c0-0cb4119059a5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:36.245062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50861a77-4462-4c07-96c0-0cb4119059a5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:36.245161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50861a77-4462-4c07-96c0-0cb4119059a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:36.309896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-50861a77-4462-4c07-96c0-0cb4119059a5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:36.311117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-50861a77-4462-4c07-96c0-0cb4119059a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:36.359039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:36.407711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:36.444160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc54c5-45c5-4a4b-8ee3-a7b492e67457 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:36.445665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b0bc54c5-45c5-4a4b-8ee3-a7b492e67457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:36.546125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc54c5-45c5-4a4b-8ee3-a7b492e67457 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:36.593193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:36.618518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9ea14-dcfa-4fbb-8121-016c9a4a8025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:36.619196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9ea14-dcfa-4fbb-8121-016c9a4a8025 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:36.619508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9ea14-dcfa-4fbb-8121-016c9a4a8025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:36.619782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9ea14-dcfa-4fbb-8121-016c9a4a8025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:36.621935 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8ea9ea14-dcfa-4fbb-8121-016c9a4a8025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:36.727504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9ea14-dcfa-4fbb-8121-016c9a4a8025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:36.728933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9ea14-dcfa-4fbb-8121-016c9a4a8025 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:36.747642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 736/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:19:46.405887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=22 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:46.744817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=66 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:56.459981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4b1f1-050f-4f40-9782-6375f168e25d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:56.460522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4b1f1-050f-4f40-9782-6375f168e25d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:56.460522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4b1f1-050f-4f40-9782-6375f168e25d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:56.461897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 735/1471] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:56.467054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214af2d-5281-4b6f-976b-b7805d09cd24 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:56.467540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214af2d-5281-4b6f-976b-b7805d09cd24 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:56.467659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7214af2d-5281-4b6f-976b-b7805d09cd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:56.530910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7214af2d-5281-4b6f-976b-b7805d09cd24 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:56.532140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7214af2d-5281-4b6f-976b-b7805d09cd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:56.629415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 737/1472] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:56.712903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e96b1-5169-46c8-ae64-e46261052b39 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:56.713163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e96b1-5169-46c8-ae64-e46261052b39 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:56.713441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e96b1-5169-46c8-ae64-e46261052b39 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:56.715481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 736/1473] 10.222.0.26 () {54 vars in 962 bytes} [Mon Jul 11 11:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:56.720292 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596db475-d828-4b09-b855-9e2b7e145f29 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:56.720589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596db475-d828-4b09-b855-9e2b7e145f29 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:56.720869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596db475-d828-4b09-b855-9e2b7e145f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:56.795265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-596db475-d828-4b09-b855-9e2b7e145f29 None None] MFA Rules not processed for user `e939bc73c40140cb89517e918722d154`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:56.796473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-596db475-d828-4b09-b855-9e2b7e145f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:56.816614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:56.871542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 738/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Mon Jul 11 11:19:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:56.878955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e96b1-5169-46c8-ae64-e46261052b39 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:56.880463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0d9e96b1-5169-46c8-ae64-e46261052b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:56.983603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e96b1-5169-46c8-ae64-e46261052b39 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:57.054241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb16d6-829f-41ab-a9fb-744745062fcd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e939bc73c40140cb89517e918722d154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:57.055185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb16d6-829f-41ab-a9fb-744745062fcd service glance] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:57.055503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb16d6-829f-41ab-a9fb-744745062fcd service glance] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:57.055787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb16d6-829f-41ab-a9fb-744745062fcd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:57.057440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cbdb16d6-829f-41ab-a9fb-744745062fcd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:57.111054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:57.164305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb16d6-829f-41ab-a9fb-744745062fcd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:57.165731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb16d6-829f-41ab-a9fb-744745062fcd service glance] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:57.184192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 737/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:19:59.307664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eab28-de5d-4aa4-8d26-1b101d0fccac None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:59.308804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eab28-de5d-4aa4-8d26-1b101d0fccac None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:59.308804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54eab28-de5d-4aa4-8d26-1b101d0fccac None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:59.310211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 739/1476] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:19:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:19:59.315453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371ff2-f69d-4c88-871b-9b73df522447 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:59.315841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371ff2-f69d-4c88-871b-9b73df522447 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:59.316015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371ff2-f69d-4c88-871b-9b73df522447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:59.379179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1e371ff2-f69d-4c88-871b-9b73df522447 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:19:59.380342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1e371ff2-f69d-4c88-871b-9b73df522447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:59.475844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 738/1477] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:19:59.500267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54eab28-de5d-4aa4-8d26-1b101d0fccac None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:19:59.501778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a54eab28-de5d-4aa4-8d26-1b101d0fccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:19:59.629792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54eab28-de5d-4aa4-8d26-1b101d0fccac None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:19:59.677557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:19:59.721686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad777e49-03c0-4011-939d-57e6cfd0cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:19:59.722343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad777e49-03c0-4011-939d-57e6cfd0cf07 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:19:59.722625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad777e49-03c0-4011-939d-57e6cfd0cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:19:59.722899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad777e49-03c0-4011-939d-57e6cfd0cf07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:19:59.723665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad777e49-03c0-4011-939d-57e6cfd0cf07 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:19:59.725445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-ad777e49-03c0-4011-939d-57e6cfd0cf07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:19:59.725445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 11 11:19:59.725445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: Jul 11 11:19:59.726599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad777e49-03c0-4011-939d-57e6cfd0cf07 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:19:59.746073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 740/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Mon Jul 11 11:19:59 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:20:01.827577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605b3c-d901-4d4d-bce7-f8c2582ab222 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:01.828190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605b3c-d901-4d4d-bce7-f8c2582ab222 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:01.828190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605b3c-d901-4d4d-bce7-f8c2582ab222 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:01.829516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 739/1479] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:20:01.834717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcc23b-d29e-4de6-8b2d-6e5e3269952d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:01.835169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcc23b-d29e-4de6-8b2d-6e5e3269952d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:01.835314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcc23b-d29e-4de6-8b2d-6e5e3269952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:01.897956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-38bcc23b-d29e-4de6-8b2d-6e5e3269952d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:20:01.899166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-38bcc23b-d29e-4de6-8b2d-6e5e3269952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:01.997265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 741/1480] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:20:02.040154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605b3c-d901-4d4d-bce7-f8c2582ab222 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:20:02.041653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7b605b3c-d901-4d4d-bce7-f8c2582ab222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:02.051497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:02.186506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605b3c-d901-4d4d-bce7-f8c2582ab222 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:20:02.264192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:02.295974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d7a89-5dcb-477c-8627-07e7cc954622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:20:02.296620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7a89-5dcb-477c-8627-07e7cc954622 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:02.296899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7a89-5dcb-477c-8627-07e7cc954622 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:02.297174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d7a89-5dcb-477c-8627-07e7cc954622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:02.298785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ea0d7a89-5dcb-477c-8627-07e7cc954622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:02.383661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d7a89-5dcb-477c-8627-07e7cc954622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:20:02.385232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d7a89-5dcb-477c-8627-07e7cc954622 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:20:02.403630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 740/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:20:04.614109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f5020-13aa-4acd-8390-c0855904be02 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:04.614690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f5020-13aa-4acd-8390-c0855904be02 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:04.614690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f5020-13aa-4acd-8390-c0855904be02 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:04.616065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 742/1482] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:20:04.621978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a006-65ea-45da-8c02-347a52c5436b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:04.622418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a006-65ea-45da-8c02-347a52c5436b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:04.622612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a006-65ea-45da-8c02-347a52c5436b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:04.686437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b634a006-65ea-45da-8c02-347a52c5436b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:20:04.687689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b634a006-65ea-45da-8c02-347a52c5436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:04.732067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:04.780645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 741/1483] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:20:04.817867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f5020-13aa-4acd-8390-c0855904be02 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:20:04.819523 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2b3f5020-13aa-4acd-8390-c0855904be02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:04.841668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:04.919521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f5020-13aa-4acd-8390-c0855904be02 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:20:04.990353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7ce7c-117d-492a-8885-930f9de58f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:20:04.990993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7ce7c-117d-492a-8885-930f9de58f33 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:04.991298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7ce7c-117d-492a-8885-930f9de58f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:04.991579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7ce7c-117d-492a-8885-930f9de58f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:04.993204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-24c7ce7c-117d-492a-8885-930f9de58f33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:05.065991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:05.101490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7ce7c-117d-492a-8885-930f9de58f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:20:05.102934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7ce7c-117d-492a-8885-930f9de58f33 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:20:05.121083 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 743/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:20:07.206964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c4686-a380-42e8-ab5b-80c86c2cf4fa None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:07.207555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c4686-a380-42e8-ab5b-80c86c2cf4fa None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:07.207555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c4686-a380-42e8-ab5b-80c86c2cf4fa None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:07.208961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 742/1485] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:20:07.214308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b494cd-e6e3-4c39-a4da-455828d55449 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:07.214754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b494cd-e6e3-4c39-a4da-455828d55449 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:07.214856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b494cd-e6e3-4c39-a4da-455828d55449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:07.278308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b3b494cd-e6e3-4c39-a4da-455828d55449 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:20:07.279522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b3b494cd-e6e3-4c39-a4da-455828d55449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:07.374638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 744/1486] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:20:07.412811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c4686-a380-42e8-ab5b-80c86c2cf4fa None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:20:07.415684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a84c4686-a380-42e8-ab5b-80c86c2cf4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:07.517153 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c4686-a380-42e8-ab5b-80c86c2cf4fa None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:20:07.588071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:07.589956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71f73a-6ddf-42b7-812d-e2e9c84c81e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:20:07.590596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f73a-6ddf-42b7-812d-e2e9c84c81e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:07.591859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f73a-6ddf-42b7-812d-e2e9c84c81e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:07.591859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f73a-6ddf-42b7-812d-e2e9c84c81e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:07.593019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cf71f73a-6ddf-42b7-812d-e2e9c84c81e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:07.703117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71f73a-6ddf-42b7-812d-e2e9c84c81e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:20:07.704592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71f73a-6ddf-42b7-812d-e2e9c84c81e2 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:20:07.722888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 743/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:20:09.837424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4c27-c811-44f7-a7d6-5f1505739d3a None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:09.837961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4c27-c811-44f7-a7d6-5f1505739d3a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:09.837961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4c27-c811-44f7-a7d6-5f1505739d3a None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:09.839313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 745/1488] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:20:09.844579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03dd35-6d05-4a19-a7f7-b213ca50270f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:09.844982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03dd35-6d05-4a19-a7f7-b213ca50270f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:09.845147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03dd35-6d05-4a19-a7f7-b213ca50270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:09.908534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9e03dd35-6d05-4a19-a7f7-b213ca50270f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:20:09.909745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9e03dd35-6d05-4a19-a7f7-b213ca50270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:09.961240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:10.004587 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 744/1489] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:20:10.044812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d4c27-c811-44f7-a7d6-5f1505739d3a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:20:10.046348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7d6d4c27-c811-44f7-a7d6-5f1505739d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:10.083940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:10.151455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d4c27-c811-44f7-a7d6-5f1505739d3a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:20:10.341623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e678af-361d-4c9c-a9d2-535b83a5bd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:20:10.342257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e678af-361d-4c9c-a9d2-535b83a5bd68 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:10.342533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e678af-361d-4c9c-a9d2-535b83a5bd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:10.342803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e678af-361d-4c9c-a9d2-535b83a5bd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:10.344438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-31e678af-361d-4c9c-a9d2-535b83a5bd68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:10.420113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:10.448544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e678af-361d-4c9c-a9d2-535b83a5bd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:20:10.449975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e678af-361d-4c9c-a9d2-535b83a5bd68 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:20:10.468347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 746/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:20:12.892384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0bbd-286c-4b79-ac99-d66b4c7aeb9d None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:12.892967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0bbd-286c-4b79-ac99-d66b4c7aeb9d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:12.892967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c0bbd-286c-4b79-ac99-d66b4c7aeb9d None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:12.894314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 745/1491] 10.222.0.26 () {54 vars in 926 bytes} [Mon Jul 11 11:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:20:12.899466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bce8d2-29fa-468c-af09-1f77dc796974 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:12.899903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bce8d2-29fa-468c-af09-1f77dc796974 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:12.900016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bce8d2-29fa-468c-af09-1f77dc796974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:12.963090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-46bce8d2-29fa-468c-af09-1f77dc796974 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:20:12.964906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-46bce8d2-29fa-468c-af09-1f77dc796974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:13.058273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 747/1492] 10.222.0.26 () {58 vars in 1023 bytes} [Mon Jul 11 11:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:20:13.097697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0bbd-286c-4b79-ac99-d66b4c7aeb9d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:20:13.099243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-989c0bbd-286c-4b79-ac99-d66b4c7aeb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:13.199928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c0bbd-286c-4b79-ac99-d66b4c7aeb9d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:20:13.271433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76b909-e609-4887-a842-4a3e93be0177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:20:13.272080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b909-e609-4887-a842-4a3e93be0177 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:20:13.272369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b909-e609-4887-a842-4a3e93be0177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:20:13.272651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b909-e609-4887-a842-4a3e93be0177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:20:13.274296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ad76b909-e609-4887-a842-4a3e93be0177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:20:13.286223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:13.388622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76b909-e609-4887-a842-4a3e93be0177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:20:13.390124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76b909-e609-4887-a842-4a3e93be0177 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:20:13.408855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 746/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:20:23.056443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=82 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:20:23.406373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=50 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:56.151382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f75ca6-b195-418a-9390-925d23f313c6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:56.151961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f75ca6-b195-418a-9390-925d23f313c6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:56.151961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f75ca6-b195-418a-9390-925d23f313c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:56.224632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-93f75ca6-b195-418a-9390-925d23f313c6 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:21:56.225874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-93f75ca6-b195-418a-9390-925d23f313c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:56.325851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 748/1494] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:21:56.334368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50988-43ed-4a4d-843a-6fbcf8f2b5cd None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:56.334937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50988-43ed-4a4d-843a-6fbcf8f2b5cd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:56.334937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50988-43ed-4a4d-843a-6fbcf8f2b5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:56.407702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cbe50988-43ed-4a4d-843a-6fbcf8f2b5cd None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:21:56.408917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cbe50988-43ed-4a4d-843a-6fbcf8f2b5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:56.502580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 747/1495] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:21:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:21:56.507757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f75ca6-b195-418a-9390-925d23f313c6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:56.508933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-93f75ca6-b195-418a-9390-925d23f313c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:56.520324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f75ca6-b195-418a-9390-925d23f313c6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:56.580149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:56.612689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:56.613342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:56.613625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:56.613903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:56.614626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:21:56.614934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:56.615962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:21:56.615962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 11 11:21:56.615962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: Jul 11 11:21:56.616849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:56.627714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 749/1496] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:21:56.636375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe50988-43ed-4a4d-843a-6fbcf8f2b5cd None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:56.645367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe50988-43ed-4a4d-843a-6fbcf8f2b5cd None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:56.705553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:56.738201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:56.738887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:56.739171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:56.739481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:56.740177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:56.742459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:21:56.742459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 11 11:21:56.742459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: Jul 11 11:21:56.743717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:56.851486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 748/1497] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:21:56 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:21:56.855484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:56.857037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:56.890210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69194068-393b-4fa7-b4e1-a663566d49a4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:56.981893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:56.982552 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:56.982832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:56.983104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:56.983830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:56.984835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:21:56.984835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 11 11:21:56.984835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: Jul 11 11:21:56.986031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:57.091157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 750/1498] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:21:56 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:21:57.095298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:57.097953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:57.160132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f420f-188f-4be0-a0d6-9ba40b553b50 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:57.244035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:57.372290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:57.372953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:57.377101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:57.377101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:57.377101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:57.377101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:21:57.377101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 11 11:21:57.377101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: Jul 11 11:21:57.377101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:57.388890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 749/1499] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:21:57.392993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:57.394495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:57.406367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fc6fa-9897-401a-b41e-51ac5b9034ba admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:57.431772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:57.499185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6290-a50a-46c5-916c-cae4d0289ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:57.499878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6290-a50a-46c5-916c-cae4d0289ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:57.500163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6290-a50a-46c5-916c-cae4d0289ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:57.500444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6290-a50a-46c5-916c-cae4d0289ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:57.501165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c6290-a50a-46c5-916c-cae4d0289ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:57.502164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-990c6290-a50a-46c5-916c-cae4d0289ad5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:21:57.502164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 11 11:21:57.502164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: Jul 11 11:21:57.503056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c6290-a50a-46c5-916c-cae4d0289ad5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:57.515176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 751/1500] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:21:57.519415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:57.520918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:57.554401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9d4f3-7312-4d5a-aaa5-445476ea92fe admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:57.643974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdb619-9943-4ffe-a66d-986a2c48537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:57.644856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdb619-9943-4ffe-a66d-986a2c48537c admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:57.645138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdb619-9943-4ffe-a66d-986a2c48537c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:57.645414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdb619-9943-4ffe-a66d-986a2c48537c admin admin] PATH_INFO: `/v3/projects/40fe99de49f640ba89165cf38ed51cb1/users/684b467ba7da4cbeae7403fd9bd65af7/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:57.668094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdb619-9943-4ffe-a66d-986a2c48537c admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fe99de49f640ba89165cf38ed51cb1, user_id=684b467ba7da4cbeae7403fd9bd65af7, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:57.669119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-cdfdb619-9943-4ffe-a66d-986a2c48537c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:21:57.669119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 11 11:21:57.669119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: Jul 11 11:21:57.671030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdb619-9943-4ffe-a66d-986a2c48537c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:57.759132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 750/1501] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:21:57 2022] PUT /identity/v3/projects/40fe99de49f640ba89165cf38ed51cb1/users/684b467ba7da4cbeae7403fd9bd65af7/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:21:57.771133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a532a-259d-4650-9041-fde5d8811919 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:57.771452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a532a-259d-4650-9041-fde5d8811919 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:57.771728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a532a-259d-4650-9041-fde5d8811919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:57.825744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5d5a532a-259d-4650-9041-fde5d8811919 None None] MFA Rules not processed for user `684b467ba7da4cbeae7403fd9bd65af7`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:21:57.826929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5d5a532a-259d-4650-9041-fde5d8811919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:57.903161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 752/1502] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:21:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:21:57.916038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826802-0e2a-4713-a2dd-be7e38122b6c None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:57.916331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826802-0e2a-4713-a2dd-be7e38122b6c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:57.916607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826802-0e2a-4713-a2dd-be7e38122b6c None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:57.917955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 751/1503] 10.222.0.26 () {54 vars in 963 bytes} [Mon Jul 11 11:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:21:57.922585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f8a53-d98c-4ff3-b95f-19d8f11f22b9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:57.922880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f8a53-d98c-4ff3-b95f-19d8f11f22b9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:57.923190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f8a53-d98c-4ff3-b95f-19d8f11f22b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:57.952500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:57.995207 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b03f8a53-d98c-4ff3-b95f-19d8f11f22b9 None None] MFA Rules not processed for user `d7cebda9ae4e4a6b9a9bbface3e7b232`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:21:57.996436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b03f8a53-d98c-4ff3-b95f-19d8f11f22b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:58.071021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 753/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Mon Jul 11 11:21:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:21:58.078348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63826802-0e2a-4713-a2dd-be7e38122b6c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:58.079867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-63826802-0e2a-4713-a2dd-be7e38122b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:58.128431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:58.179970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63826802-0e2a-4713-a2dd-be7e38122b6c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:58.251805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:58.252443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:58.252727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:58.253005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:58.254628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:58.352594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:58.353989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:58.357577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:58.371663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 752/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:21:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:21:58.413426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010d498-6f3e-471c-b266-25a7bf6edfb3 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:58.414782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010d498-6f3e-471c-b266-25a7bf6edfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:58.414782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010d498-6f3e-471c-b266-25a7bf6edfb3 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:58.415300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 754/1506] 10.222.0.26 () {52 vars in 829 bytes} [Mon Jul 11 11:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:21:58.418924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:58.420156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:58.521985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4dd87-92cf-441e-a82f-7a5e2c0d1444 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:58.611679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:58.612466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:58.612744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:58.613011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] PATH_INFO: `/v3/users/684b467ba7da4cbeae7403fd9bd65af7` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:58.625232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] RBAC: Authorizing `identity:delete_user(user_id=684b467ba7da4cbeae7403fd9bd65af7)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:58.626234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:21:58.626234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 11 11:21:58.626234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: Jul 11 11:21:58.627462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:58.636741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:59.106593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684b467ba7da4cbeae7403fd9bd65af7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:21:59.107142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684b467ba7da4cbeae7403fd9bd65af7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:21:59.107592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684b467ba7da4cbeae7403fd9bd65af7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:21:59.119889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684b467ba7da4cbeae7403fd9bd65af7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:21:59.137498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684b467ba7da4cbeae7403fd9bd65af7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:21:59.219043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 753/1507] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:21:58 2022] DELETE /identity/v3/users/684b467ba7da4cbeae7403fd9bd65af7 => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:21:59.223460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdecb6-0a6d-405e-9cac-5d9a1dc1219a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:59.226830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdecb6-0a6d-405e-9cac-5d9a1dc1219a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:59.226830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cdecb6-0a6d-405e-9cac-5d9a1dc1219a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:59.279412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-61cdecb6-0a6d-405e-9cac-5d9a1dc1219a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:21:59.280636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-61cdecb6-0a6d-405e-9cac-5d9a1dc1219a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:59.354696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:59.366473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 755/1508] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:21:59.377833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:21:59.379071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:59.479348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b26f-ff07-4e24-890e-5e1d4d54a93f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:21:59.531878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:21:59.550224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e498eb-30e7-4f92-807f-52d89a8810ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:21:59.550869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e498eb-30e7-4f92-807f-52d89a8810ab service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:21:59.551156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e498eb-30e7-4f92-807f-52d89a8810ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:21:59.551459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e498eb-30e7-4f92-807f-52d89a8810ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:21:59.553108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f9e498eb-30e7-4f92-807f-52d89a8810ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:21:59.660110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e498eb-30e7-4f92-807f-52d89a8810ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:21:59.661459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e498eb-30e7-4f92-807f-52d89a8810ab service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:21:59.680527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 754/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:22:00.175879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdecb6-0a6d-405e-9cac-5d9a1dc1219a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:22:00.177790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-61cdecb6-0a6d-405e-9cac-5d9a1dc1219a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:22:00.189864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cdecb6-0a6d-405e-9cac-5d9a1dc1219a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:22:00.286381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:22:00.287238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:22:00.288102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:22:00.290167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] PATH_INFO: `/v3/projects/40fe99de49f640ba89165cf38ed51cb1` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:22:00.302226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fe99de49f640ba89165cf38ed51cb1)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:22:00.304002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:22:00.304002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 11 11:22:00.304002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: Jul 11 11:22:00.305220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:22:00.426700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fe99de49f640ba89165cf38ed51cb1', 'request_id': 'req-89dd00f5-210c-4d3c-9158-e04d16125ffe'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:22:00.433209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fe99de49f640ba89165cf38ed51cb1', 'request_id': 'req-89dd00f5-210c-4d3c-9158-e04d16125ffe'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:22:00.519117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fe99de49f640ba89165cf38ed51cb1', 'request_id': 'req-89dd00f5-210c-4d3c-9158-e04d16125ffe'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:22:00.549493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] The token cache is being invalidate because project 40fe99de49f640ba89165cf38ed51cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:22:00.549782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:22:00.550299 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89dd00f5-210c-4d3c-9158-e04d16125ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:22:00.551693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 756/1510] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:22:00 2022] DELETE /identity/v3/projects/40fe99de49f640ba89165cf38ed51cb1 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:22:09.677496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=62 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:22:10.547542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:23:40.998365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e2002-5122-47f6-beb6-33ba05ccab1f None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:40.998977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e2002-5122-47f6-beb6-33ba05ccab1f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:40.998977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e2002-5122-47f6-beb6-33ba05ccab1f None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:41.000408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:23:41.006226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53227280-aa0c-4437-9e7c-3ab659b0a3f4 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:41.006765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53227280-aa0c-4437-9e7c-3ab659b0a3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:41.006765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53227280-aa0c-4437-9e7c-3ab659b0a3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:41.085258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-53227280-aa0c-4437-9e7c-3ab659b0a3f4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:23:41.086476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-53227280-aa0c-4437-9e7c-3ab659b0a3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:41.186839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:23:41.339081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e2002-5122-47f6-beb6-33ba05ccab1f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:23:41.341343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5c9e2002-5122-47f6-beb6-33ba05ccab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:41.457720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e2002-5122-47f6-beb6-33ba05ccab1f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:23:41.463018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d07d03-51a7-4673-bb84-dd32e66dffc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:23:41.463698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d07d03-51a7-4673-bb84-dd32e66dffc9 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:41.463989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d07d03-51a7-4673-bb84-dd32e66dffc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:41.464269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d07d03-51a7-4673-bb84-dd32e66dffc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:41.472330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d07d03-51a7-4673-bb84-dd32e66dffc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:23:41.473904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d07d03-51a7-4673-bb84-dd32e66dffc9 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:23:41.483404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 756/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:23:42.308921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4de74-737c-4c52-a8a1-ae7bd7cfcb91 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:42.309503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4de74-737c-4c52-a8a1-ae7bd7cfcb91 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:42.309503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4de74-737c-4c52-a8a1-ae7bd7cfcb91 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:42.310912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:23:42.315795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ae4e4-4df6-43ce-a26b-f57f90621c80 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:42.316251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ae4e4-4df6-43ce-a26b-f57f90621c80 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:42.316373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ae4e4-4df6-43ce-a26b-f57f90621c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:42.335113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c47ae4e4-4df6-43ce-a26b-f57f90621c80 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:23:42.336276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c47ae4e4-4df6-43ce-a26b-f57f90621c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:42.343968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:23:42.496837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4de74-737c-4c52-a8a1-ae7bd7cfcb91 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:23:42.511027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4de74-737c-4c52-a8a1-ae7bd7cfcb91 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:23:42.516440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb83ee-b545-4829-90f5-6fa6e68a52a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:23:42.517074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb83ee-b545-4829-90f5-6fa6e68a52a4 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:42.517362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb83ee-b545-4829-90f5-6fa6e68a52a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:42.517644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb83ee-b545-4829-90f5-6fa6e68a52a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:42.525639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb83ee-b545-4829-90f5-6fa6e68a52a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:23:42.527223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb83ee-b545-4829-90f5-6fa6e68a52a4 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:23:42.536615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 759/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:23:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:23:43.093384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed80c4-31b2-4a26-b5a7-db8ba1676a84 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:43.093671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed80c4-31b2-4a26-b5a7-db8ba1676a84 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:43.093951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed80c4-31b2-4a26-b5a7-db8ba1676a84 None None] PATH_INFO: `/v3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:43.095352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 758/1517] 10.222.0.46 () {54 vars in 925 bytes} [Mon Jul 11 11:23:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:23:43.114405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e37e5-972c-45e1-94b6-c417b1b7fd2f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:43.114697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e37e5-972c-45e1-94b6-c417b1b7fd2f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:43.115025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e37e5-972c-45e1-94b6-c417b1b7fd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:43.170957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a96e37e5-972c-45e1-94b6-c417b1b7fd2f None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:23:43.172175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a96e37e5-972c-45e1-94b6-c417b1b7fd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:43.188532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:23:43.243358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 760/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Jul 11 11:23:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:23:47.212128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb6030-94d4-401b-803d-212d315abb27 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:47.212675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb6030-94d4-401b-803d-212d315abb27 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:47.212675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb6030-94d4-401b-803d-212d315abb27 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:47.214013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 759/1519] 10.222.0.26 () {54 vars in 960 bytes} [Mon Jul 11 11:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:23:47.219050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4596a33-d296-4a72-8ce5-3be76f2fbc93 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:47.219502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4596a33-d296-4a72-8ce5-3be76f2fbc93 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:47.219764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4596a33-d296-4a72-8ce5-3be76f2fbc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:47.237910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f4596a33-d296-4a72-8ce5-3be76f2fbc93 None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:23:47.239099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f4596a33-d296-4a72-8ce5-3be76f2fbc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:47.247506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 761/1520] 10.222.0.26 () {58 vars in 1057 bytes} [Mon Jul 11 11:23:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:23:47.254917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb6030-94d4-401b-803d-212d315abb27 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:23:47.267811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb6030-94d4-401b-803d-212d315abb27 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:23:47.273192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fbe111-66b8-4957-a4e6-f4014a1cef34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:23:47.273813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbe111-66b8-4957-a4e6-f4014a1cef34 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:47.274093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbe111-66b8-4957-a4e6-f4014a1cef34 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:47.274363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fbe111-66b8-4957-a4e6-f4014a1cef34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:47.282378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbe111-66b8-4957-a4e6-f4014a1cef34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:23:47.283972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fbe111-66b8-4957-a4e6-f4014a1cef34 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:23:47.293956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 760/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:23:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:23:53.842884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106c94-7d6e-43c3-9136-7783da7fdf63 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:53.843496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106c94-7d6e-43c3-9136-7783da7fdf63 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:53.843496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106c94-7d6e-43c3-9136-7783da7fdf63 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:53.844829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:23:53.850307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615db4b-e456-4b56-9460-cd2b2b01dc8f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:53.850733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615db4b-e456-4b56-9460-cd2b2b01dc8f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:53.850873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615db4b-e456-4b56-9460-cd2b2b01dc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:53.869311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8615db4b-e456-4b56-9460-cd2b2b01dc8f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:23:53.870526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8615db4b-e456-4b56-9460-cd2b2b01dc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:53.878279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:23:54.029517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106c94-7d6e-43c3-9136-7783da7fdf63 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:23:54.044627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106c94-7d6e-43c3-9136-7783da7fdf63 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:23:54.050160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3456673-e162-44b6-b974-fa36af7c09c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:23:54.057472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3456673-e162-44b6-b974-fa36af7c09c0 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:54.057472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3456673-e162-44b6-b974-fa36af7c09c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:54.057472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3456673-e162-44b6-b974-fa36af7c09c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:54.060100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3456673-e162-44b6-b974-fa36af7c09c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:23:54.061681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3456673-e162-44b6-b974-fa36af7c09c0 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:23:54.071231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 763/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:23:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:23:54.894422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c573e3-a0a6-4310-b714-a1b48db50cb4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:54.895005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c573e3-a0a6-4310-b714-a1b48db50cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:54.895005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c573e3-a0a6-4310-b714-a1b48db50cb4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:54.896869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 11:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:23:54.901737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e9ec-1fe6-4443-85e6-15387ad917d5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:54.902192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e9ec-1fe6-4443-85e6-15387ad917d5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:54.902304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e9ec-1fe6-4443-85e6-15387ad917d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:54.920331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f770e9ec-1fe6-4443-85e6-15387ad917d5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:23:54.921588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f770e9ec-1fe6-4443-85e6-15387ad917d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:54.929984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 11:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:23:55.083063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c573e3-a0a6-4310-b714-a1b48db50cb4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:23:55.097869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c573e3-a0a6-4310-b714-a1b48db50cb4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:23:55.103233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf45ca0-b6c4-4c36-8b63-9ba5c1746fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:23:55.103895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf45ca0-b6c4-4c36-8b63-9ba5c1746fba service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:55.104181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf45ca0-b6c4-4c36-8b63-9ba5c1746fba service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:55.104459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf45ca0-b6c4-4c36-8b63-9ba5c1746fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:55.112491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf45ca0-b6c4-4c36-8b63-9ba5c1746fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:23:55.114067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf45ca0-b6c4-4c36-8b63-9ba5c1746fba service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:23:55.123441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 763/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:23:58.928087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b67d-e42b-49b8-b0b4-2879bb605dfe None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:58.928667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b67d-e42b-49b8-b0b4-2879bb605dfe None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:58.928667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b67d-e42b-49b8-b0b4-2879bb605dfe None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:58.929988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:23:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:23:58.935385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4cc2c-668b-4f44-9852-17c792dec2d7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:58.935843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4cc2c-668b-4f44-9852-17c792dec2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:58.935940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4cc2c-668b-4f44-9852-17c792dec2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:58.953464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-acf4cc2c-668b-4f44-9852-17c792dec2d7 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:23:58.954655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-acf4cc2c-668b-4f44-9852-17c792dec2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:58.962422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:23:59.035306 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4b67d-e42b-49b8-b0b4-2879bb605dfe None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:23:59.037767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fdb4b67d-e42b-49b8-b0b4-2879bb605dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:23:59.117254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4b67d-e42b-49b8-b0b4-2879bb605dfe None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:23:59.120764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638eee2-4b54-4cd4-b7df-3cb9b17791b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:23:59.121400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638eee2-4b54-4cd4-b7df-3cb9b17791b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:23:59.121680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638eee2-4b54-4cd4-b7df-3cb9b17791b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:23:59.121954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638eee2-4b54-4cd4-b7df-3cb9b17791b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:23:59.130568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638eee2-4b54-4cd4-b7df-3cb9b17791b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:23:59.132021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638eee2-4b54-4cd4-b7df-3cb9b17791b0 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:23:59.141405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 766/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:24:01.149717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041068c-ef3c-4975-8c11-1a3385ce771e None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:01.150234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041068c-ef3c-4975-8c11-1a3385ce771e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:01.150393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041068c-ef3c-4975-8c11-1a3385ce771e None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:01.151663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:24:01.156721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff97c16-fe30-4545-92b8-a59317eb4c0d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:01.157126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff97c16-fe30-4545-92b8-a59317eb4c0d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:01.157286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff97c16-fe30-4545-92b8-a59317eb4c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:01.175654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9ff97c16-fe30-4545-92b8-a59317eb4c0d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:24:01.176924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9ff97c16-fe30-4545-92b8-a59317eb4c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:24:01.184994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:24:01.220686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041068c-ef3c-4975-8c11-1a3385ce771e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:24:01.231202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041068c-ef3c-4975-8c11-1a3385ce771e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:24:01.234701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d208b-fe3c-41c8-a61a-34d6b4e80301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:24:01.235363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d208b-fe3c-41c8-a61a-34d6b4e80301 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:01.235654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d208b-fe3c-41c8-a61a-34d6b4e80301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:01.235936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d208b-fe3c-41c8-a61a-34d6b4e80301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:01.244835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d208b-fe3c-41c8-a61a-34d6b4e80301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:24:01.246252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d208b-fe3c-41c8-a61a-34d6b4e80301 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:24:01.256466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 766/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:24:05.201804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42d695-608c-4bc4-bf1f-110a54910a3b None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:05.202413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42d695-608c-4bc4-bf1f-110a54910a3b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:05.202413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42d695-608c-4bc4-bf1f-110a54910a3b None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:05.203788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:24:05.209646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcc36-9db6-42e9-8de8-acb46a292818 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:05.210080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcc36-9db6-42e9-8de8-acb46a292818 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:05.210260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfcc36-9db6-42e9-8de8-acb46a292818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:05.227716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-62dfcc36-9db6-42e9-8de8-acb46a292818 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:24:05.228921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-62dfcc36-9db6-42e9-8de8-acb46a292818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:24:05.236760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:24:05.540330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42d695-608c-4bc4-bf1f-110a54910a3b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:24:05.542004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fa42d695-608c-4bc4-bf1f-110a54910a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:24:05.605052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:24:05.617866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42d695-608c-4bc4-bf1f-110a54910a3b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:24:05.621473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc43c4-f495-4f40-bfe9-f40583299a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:24:05.622118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc43c4-f495-4f40-bfe9-f40583299a9f service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:05.622400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc43c4-f495-4f40-bfe9-f40583299a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:05.622678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc43c4-f495-4f40-bfe9-f40583299a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:05.632038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc43c4-f495-4f40-bfe9-f40583299a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:24:05.633539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc43c4-f495-4f40-bfe9-f40583299a9f service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:24:05.643846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 769/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:24:07.673413 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cac21-bda6-4e5a-9904-3aa13210b6b4 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:07.673986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cac21-bda6-4e5a-9904-3aa13210b6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:07.673986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86cac21-bda6-4e5a-9904-3aa13210b6b4 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:07.675356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 11:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:24:07.680826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac9d15-fbd4-4890-9179-ca8ec4a79c0a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:07.681278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac9d15-fbd4-4890-9179-ca8ec4a79c0a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:07.681395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac9d15-fbd4-4890-9179-ca8ec4a79c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:07.698891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-55ac9d15-fbd4-4890-9179-ca8ec4a79c0a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:24:07.700110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-55ac9d15-fbd4-4890-9179-ca8ec4a79c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:24:07.708324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:24:08.004998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cac21-bda6-4e5a-9904-3aa13210b6b4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:24:08.006652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c86cac21-bda6-4e5a-9904-3aa13210b6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:24:08.019428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86cac21-bda6-4e5a-9904-3aa13210b6b4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:24:08.023489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c41fe-aee7-43a4-9e54-e3b217e4c1d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:24:08.024122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c41fe-aee7-43a4-9e54-e3b217e4c1d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:24:08.024406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c41fe-aee7-43a4-9e54-e3b217e4c1d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:24:08.024683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c41fe-aee7-43a4-9e54-e3b217e4c1d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:24:08.033467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c41fe-aee7-43a4-9e54-e3b217e4c1d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:24:08.034878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c41fe-aee7-43a4-9e54-e3b217e4c1d8 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:24:08.044188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 769/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:24:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:24:17.694129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=11 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:24:18.040133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:36.414894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201f8a-6ab0-4885-9f99-eddb7a5c5aba None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.416291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201f8a-6ab0-4885-9f99-eddb7a5c5aba None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.416801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88201f8a-6ab0-4885-9f99-eddb7a5c5aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.450768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-88201f8a-6ab0-4885-9f99-eddb7a5c5aba None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.452022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-88201f8a-6ab0-4885-9f99-eddb7a5c5aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.464035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 771/1540] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:36.473553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08ae9-6e3f-47f3-b2b6-e66b48f8acce None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.474102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08ae9-6e3f-47f3-b2b6-e66b48f8acce None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.474102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08ae9-6e3f-47f3-b2b6-e66b48f8acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.499091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0ba08ae9-6e3f-47f3-b2b6-e66b48f8acce None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.500333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0ba08ae9-6e3f-47f3-b2b6-e66b48f8acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.513692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 770/1541] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:36.518641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88201f8a-6ab0-4885-9f99-eddb7a5c5aba None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:36.533598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88201f8a-6ab0-4885-9f99-eddb7a5c5aba None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:36.540150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f996d-b546-43f6-8382-16495ed27e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:36.540852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f996d-b546-43f6-8382-16495ed27e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.541137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f996d-b546-43f6-8382-16495ed27e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.541416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f996d-b546-43f6-8382-16495ed27e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.542155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f996d-b546-43f6-8382-16495ed27e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:36.542475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f996d-b546-43f6-8382-16495ed27e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:36.544141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f996d-b546-43f6-8382-16495ed27e99 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:36.561862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 772/1542] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:36.570844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08ae9-6e3f-47f3-b2b6-e66b48f8acce None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:36.585520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08ae9-6e3f-47f3-b2b6-e66b48f8acce None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:36.590207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4da61-2c0f-413c-9806-24f5ee9f17aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:36.590884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4da61-2c0f-413c-9806-24f5ee9f17aa admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.591170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4da61-2c0f-413c-9806-24f5ee9f17aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.591652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4da61-2c0f-413c-9806-24f5ee9f17aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.592373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4da61-2c0f-413c-9806-24f5ee9f17aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:36.594285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4da61-2c0f-413c-9806-24f5ee9f17aa admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:36.679750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058fbd2-6c80-4844-b65b-2049f4035386 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.680582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058fbd2-6c80-4844-b65b-2049f4035386 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.680582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058fbd2-6c80-4844-b65b-2049f4035386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.705326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 771/1543] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/projects => generated 391 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:36.706763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ec6f1-b96c-41e4-bf30-6060d5f0fb3c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.707053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ec6f1-b96c-41e4-bf30-6060d5f0fb3c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.707369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ec6f1-b96c-41e4-bf30-6060d5f0fb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.708159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f058fbd2-6c80-4844-b65b-2049f4035386 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.709382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f058fbd2-6c80-4844-b65b-2049f4035386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.739194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-700ec6f1-b96c-41e4-bf30-6060d5f0fb3c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.740476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-700ec6f1-b96c-41e4-bf30-6060d5f0fb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.823557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 773/1544] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:36.825353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058fbd2-6c80-4844-b65b-2049f4035386 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:36.841146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058fbd2-6c80-4844-b65b-2049f4035386 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:36.847113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5454e-17e7-4ffc-a9d1-a3cc45fdcc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:36.847837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5454e-17e7-4ffc-a9d1-a3cc45fdcc94 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.848125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5454e-17e7-4ffc-a9d1-a3cc45fdcc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.848564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5454e-17e7-4ffc-a9d1-a3cc45fdcc94 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.849274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5454e-17e7-4ffc-a9d1-a3cc45fdcc94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:36.851480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5454e-17e7-4ffc-a9d1-a3cc45fdcc94 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:36.856784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 772/1545] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:36.858354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4deb45-3e97-46b1-9666-c913c82d14ea None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.858762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4deb45-3e97-46b1-9666-c913c82d14ea None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.859052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4deb45-3e97-46b1-9666-c913c82d14ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.880100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8a4deb45-3e97-46b1-9666-c913c82d14ea None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.881392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8a4deb45-3e97-46b1-9666-c913c82d14ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.889683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 773/1546] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:36.890788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41668ef9-11ac-43b1-a3c3-a5ad623a4d34 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.891085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41668ef9-11ac-43b1-a3c3-a5ad623a4d34 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.891401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41668ef9-11ac-43b1-a3c3-a5ad623a4d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.909209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-41668ef9-11ac-43b1-a3c3-a5ad623a4d34 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.910436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-41668ef9-11ac-43b1-a3c3-a5ad623a4d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.918425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 774/1547] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:36.919665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f4c97-7b84-49be-b1ec-e96302ad574c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.919950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f4c97-7b84-49be-b1ec-e96302ad574c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.920350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f4c97-7b84-49be-b1ec-e96302ad574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.942760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ea3f4c97-7b84-49be-b1ec-e96302ad574c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.944200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ea3f4c97-7b84-49be-b1ec-e96302ad574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.952146 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 775/1548] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:36.953340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4745487-5f71-4f10-8502-88fe1bd9be59 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.953621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4745487-5f71-4f10-8502-88fe1bd9be59 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.953895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4745487-5f71-4f10-8502-88fe1bd9be59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.957606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 774/1549] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:36.959090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a74a-9533-4dc4-b0cd-7edb9d842d10 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.959416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a74a-9533-4dc4-b0cd-7edb9d842d10 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.959698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39a74a-9533-4dc4-b0cd-7edb9d842d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.977208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c4745487-5f71-4f10-8502-88fe1bd9be59 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.978049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7d39a74a-9533-4dc4-b0cd-7edb9d842d10 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:36.978427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c4745487-5f71-4f10-8502-88fe1bd9be59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.979216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7d39a74a-9533-4dc4-b0cd-7edb9d842d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:36.986310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 776/1550] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:36.988391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 775/1551] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:36.988843 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d138-d6ad-4b91-af09-16f6caf04f02 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.989130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d138-d6ad-4b91-af09-16f6caf04f02 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.989405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d138-d6ad-4b91-af09-16f6caf04f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:36.991599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d2c2-7800-4908-b92b-44591677e777 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:36.991893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d2c2-7800-4908-b92b-44591677e777 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:36.992177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d2c2-7800-4908-b92b-44591677e777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.007519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f2b5d138-d6ad-4b91-af09-16f6caf04f02 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:37.008714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f2b5d138-d6ad-4b91-af09-16f6caf04f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:37.009831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e4f3d2c2-7800-4908-b92b-44591677e777 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:37.010991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e4f3d2c2-7800-4908-b92b-44591677e777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:37.016566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 776/1552] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:37.017754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708a7b-843c-4e76-b468-4af1626ed60b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.018043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708a7b-843c-4e76-b468-4af1626ed60b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.018358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708a7b-843c-4e76-b468-4af1626ed60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.022457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 777/1553] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:37.022457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0c2f0-d2c9-4e69-800a-33fc79f73ed3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.022457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0c2f0-d2c9-4e69-800a-33fc79f73ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.022457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0c2f0-d2c9-4e69-800a-33fc79f73ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.039493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d9e0c2f0-d2c9-4e69-800a-33fc79f73ed3 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:37.040478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c2708a7b-843c-4e76-b468-4af1626ed60b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:37.040682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d9e0c2f0-d2c9-4e69-800a-33fc79f73ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:37.041629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c2708a7b-843c-4e76-b468-4af1626ed60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:37.048497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 778/1554] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:37.049267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0c2f0-d2c9-4e69-800a-33fc79f73ed3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.049753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 777/1555] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:37.054336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7fb6d-e416-4bca-b3fe-b2b332b03ea9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.054336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7fb6d-e416-4bca-b3fe-b2b332b03ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.054336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7fb6d-e416-4bca-b3fe-b2b332b03ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.059716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0c2f0-d2c9-4e69-800a-33fc79f73ed3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.064344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11037494-a7b8-4a9e-8fb7-867c24106927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.065012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11037494-a7b8-4a9e-8fb7-867c24106927 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.065839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11037494-a7b8-4a9e-8fb7-867c24106927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.066118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11037494-a7b8-4a9e-8fb7-867c24106927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.066852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11037494-a7b8-4a9e-8fb7-867c24106927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.070932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11037494-a7b8-4a9e-8fb7-867c24106927 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.073401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f6d7fb6d-e416-4bca-b3fe-b2b332b03ea9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:37.073989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f6d7fb6d-e416-4bca-b3fe-b2b332b03ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:37.078910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 779/1556] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:37.079749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11037494-a7b8-4a9e-8fb7-867c24106927 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.084778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 778/1557] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:37.086001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd21be-b971-477a-a94b-1fe96ff220f2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.086287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd21be-b971-477a-a94b-1fe96ff220f2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.086564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd21be-b971-477a-a94b-1fe96ff220f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.093398 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11037494-a7b8-4a9e-8fb7-867c24106927 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.098169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.098816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.099103 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.099414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.100135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:37.100448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.101449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:25:37.101449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 11 11:25:37.101449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: Jul 11 11:25:37.102340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.104301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-34dd21be-b971-477a-a94b-1fe96ff220f2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:37.105472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-34dd21be-b971-477a-a94b-1fe96ff220f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:37.113401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 779/1558] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:37.114168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd21be-b971-477a-a94b-1fe96ff220f2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.121462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 780/1559] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:37.121462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.124502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd21be-b971-477a-a94b-1fe96ff220f2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.126655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75722d8-144f-47a1-bc5c-8dfb8c92a081 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.131857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.132098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189090b-f785-421f-8ec0-e682d51c9050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.132543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.132833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.133112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.133214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189090b-f785-421f-8ec0-e682d51c9050 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.133688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189090b-f785-421f-8ec0-e682d51c9050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.133848 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:37.134161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.134271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189090b-f785-421f-8ec0-e682d51c9050 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.135510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189090b-f785-421f-8ec0-e682d51c9050 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:37.135811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.136009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189090b-f785-421f-8ec0-e682d51c9050 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.138261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189090b-f785-421f-8ec0-e682d51c9050 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.150889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 781/1560] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:37.151754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.161677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 780/1561] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:37.161677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dee4-9498-4c09-a924-8edef5f64e7f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.161677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dee4-9498-4c09-a924-8edef5f64e7f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.161677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355dee4-9498-4c09-a924-8edef5f64e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.162093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c459830-b8b0-436b-b434-3820da9229a6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.166448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.167101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.167413 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.167694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.168527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:37.168838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.170417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.176543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a355dee4-9498-4c09-a924-8edef5f64e7f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:37.178008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a355dee4-9498-4c09-a924-8edef5f64e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:37.182537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 782/1562] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:37.183500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.185912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 781/1563] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:37.186682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355dee4-9498-4c09-a924-8edef5f64e7f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.196734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355dee4-9498-4c09-a924-8edef5f64e7f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.198355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dff7-b770-4451-ad80-a93eb87320d4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.201284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.201926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.202206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.202481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.202901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a5611-f082-4574-91d8-c1d64d537714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.203195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:37.203566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.203682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a5611-f082-4574-91d8-c1d64d537714 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.203903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a5611-f082-4574-91d8-c1d64d537714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.204179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a5611-f082-4574-91d8-c1d64d537714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.204888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a5611-f082-4574-91d8-c1d64d537714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.205132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.207024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a5611-f082-4574-91d8-c1d64d537714 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.223657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 783/1564] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:37.223848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 782/1565] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:37.223848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.224038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a5611-f082-4574-91d8-c1d64d537714 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.229415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed0d7c-08f5-42c1-ab60-1449e3a3773e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.233235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a5611-f082-4574-91d8-c1d64d537714 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.233938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.234569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.234853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.235121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.235854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.236844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:25:37.236844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 11 11:25:37.236844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: Jul 11 11:25:37.237934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.238076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.238575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.238865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.239152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.239906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:37.240219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.241796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.253988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 784/1566] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:37.254907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.264354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e331b6-a931-4dfc-b8b2-d21f3cdb5218 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.269473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd0dd-5fe2-42e1-873e-82ae92e1aeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.270940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd0dd-5fe2-42e1-873e-82ae92e1aeb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.270940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd0dd-5fe2-42e1-873e-82ae92e1aeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.270940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fd0dd-5fe2-42e1-873e-82ae92e1aeb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.272336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fd0dd-5fe2-42e1-873e-82ae92e1aeb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.273467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fd0dd-5fe2-42e1-873e-82ae92e1aeb8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.279051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:37.327289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 783/1567] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:37.327289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.351453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 785/1568] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:37.351453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd0dd-5fe2-42e1-873e-82ae92e1aeb8 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.367296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:37.390519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4934b-22fc-4429-bf64-d3c7b61c220a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.411878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fd0dd-5fe2-42e1-873e-82ae92e1aeb8 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.416450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2c647-5dde-4632-af0f-7f7c458e76ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.425968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2c647-5dde-4632-af0f-7f7c458e76ff admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.425968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2c647-5dde-4632-af0f-7f7c458e76ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.425968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2c647-5dde-4632-af0f-7f7c458e76ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.425968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2c647-5dde-4632-af0f-7f7c458e76ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.425968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af2c647-5dde-4632-af0f-7f7c458e76ff admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.449035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715214-1332-4e42-ba0c-487cba6f8f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.449776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715214-1332-4e42-ba0c-487cba6f8f7d admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.450165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715214-1332-4e42-ba0c-487cba6f8f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.450459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715214-1332-4e42-ba0c-487cba6f8f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.451202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8715214-1332-4e42-ba0c-487cba6f8f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.453173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8715214-1332-4e42-ba0c-487cba6f8f7d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.511988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 786/1569] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/projects => generated 367 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:37.513129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2c647-5dde-4632-af0f-7f7c458e76ff admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.541624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 784/1570] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/projects => generated 385 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:37.541624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715214-1332-4e42-ba0c-487cba6f8f7d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.576392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2c647-5dde-4632-af0f-7f7c458e76ff admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.602375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715214-1332-4e42-ba0c-487cba6f8f7d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.606858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.607793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.608193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.608474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] PATH_INFO: `/v3/projects/c8639f8c836d4ad79d147b855d9017f8/users/e47c365946cf4264a2d2205ace04379d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.623042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8639f8c836d4ad79d147b855d9017f8, user_id=e47c365946cf4264a2d2205ace04379d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.624123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:25:37.624123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 11 11:25:37.624123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: Jul 11 11:25:37.626080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.634196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.640102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.640102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.640102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.640102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:37.640102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.640102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.649615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 787/1571] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:37.650364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.660269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66379db-c32c-4ffa-9c76-b8ac1e4f0dd5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.664802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aabecc-9818-42c5-afbd-08a704049f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.665434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aabecc-9818-42c5-afbd-08a704049f16 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.665713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aabecc-9818-42c5-afbd-08a704049f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.666004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aabecc-9818-42c5-afbd-08a704049f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.666656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aabecc-9818-42c5-afbd-08a704049f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.668511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aabecc-9818-42c5-afbd-08a704049f16 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.678782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:37.716023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 785/1572] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:37 2022] PUT /identity/v3/projects/c8639f8c836d4ad79d147b855d9017f8/users/e47c365946cf4264a2d2205ace04379d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:37.716974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.763780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 788/1573] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:37.764843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aabecc-9818-42c5-afbd-08a704049f16 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.767608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:37.778875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad78b6e-9b26-43ba-85f9-a8735b47aef3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.827923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aabecc-9818-42c5-afbd-08a704049f16 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.832441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.833108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.833433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.833715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.834250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66956b10-591c-4376-a8d6-713109eadc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.834557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.834904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956b10-591c-4376-a8d6-713109eadc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.835186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956b10-591c-4376-a8d6-713109eadc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.835496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956b10-591c-4376-a8d6-713109eadc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.835618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:25:37.835618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 11 11:25:37.835618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: Jul 11 11:25:37.836171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66956b10-591c-4376-a8d6-713109eadc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.836818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.838149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66956b10-591c-4376-a8d6-713109eadc54 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.915881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 789/1574] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:37.916757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.926328 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251dca-49a9-4a49-9147-f241cfb0c8a6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.929281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 786/1575] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:37.929281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66956b10-591c-4376-a8d6-713109eadc54 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.932675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abb45b-b0fb-42c5-8ed1-a4ab71db2320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.932675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb45b-b0fb-42c5-8ed1-a4ab71db2320 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.932675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb45b-b0fb-42c5-8ed1-a4ab71db2320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.932675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb45b-b0fb-42c5-8ed1-a4ab71db2320 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.933349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abb45b-b0fb-42c5-8ed1-a4ab71db2320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:37.935347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abb45b-b0fb-42c5-8ed1-a4ab71db2320 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:37.993247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 790/1576] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/users => generated 293 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:37.993439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66956b10-591c-4376-a8d6-713109eadc54 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:37.993529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abb45b-b0fb-42c5-8ed1-a4ab71db2320 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:37.997976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4f13f-a053-45ae-b7cb-fc7e068bcacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:37.998656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4f13f-a053-45ae-b7cb-fc7e068bcacc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:37.998945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4f13f-a053-45ae-b7cb-fc7e068bcacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:37.999226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4f13f-a053-45ae-b7cb-fc7e068bcacc admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:37.999951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4f13f-a053-45ae-b7cb-fc7e068bcacc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.000543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abb45b-b0fb-42c5-8ed1-a4ab71db2320 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.001899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4f13f-a053-45ae-b7cb-fc7e068bcacc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.005553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a936a-d79a-41c7-bbd0-50811c180db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.006195 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a936a-d79a-41c7-bbd0-50811c180db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.006480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a936a-d79a-41c7-bbd0-50811c180db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.006754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a936a-d79a-41c7-bbd0-50811c180db7 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.007494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a936a-d79a-41c7-bbd0-50811c180db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.009385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a936a-d79a-41c7-bbd0-50811c180db7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.076094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 787/1577] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/users => generated 284 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:38.076902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4f13f-a053-45ae-b7cb-fc7e068bcacc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.087104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4f13f-a053-45ae-b7cb-fc7e068bcacc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.091827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a365bf4-9efc-4c96-80ec-25c0b30a3546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.092464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a365bf4-9efc-4c96-80ec-25c0b30a3546 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.092745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a365bf4-9efc-4c96-80ec-25c0b30a3546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.093018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a365bf4-9efc-4c96-80ec-25c0b30a3546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.093217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 791/1578] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:37 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:38.093693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a365bf4-9efc-4c96-80ec-25c0b30a3546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.094523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce887e1-942b-485f-aa95-137ae3fc56f3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.094809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce887e1-942b-485f-aa95-137ae3fc56f3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.095125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce887e1-942b-485f-aa95-137ae3fc56f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.095656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a365bf4-9efc-4c96-80ec-25c0b30a3546 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.112596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cce887e1-942b-485f-aa95-137ae3fc56f3 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:38.113818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cce887e1-942b-485f-aa95-137ae3fc56f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:38.121479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 792/1579] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:38.122307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce887e1-942b-485f-aa95-137ae3fc56f3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.132709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce887e1-942b-485f-aa95-137ae3fc56f3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.137374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dcc2c-f15a-4ba3-9c40-2ba6a9c316ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.139545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dcc2c-f15a-4ba3-9c40-2ba6a9c316ab admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.139545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dcc2c-f15a-4ba3-9c40-2ba6a9c316ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.139545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dcc2c-f15a-4ba3-9c40-2ba6a9c316ab admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.139545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dcc2c-f15a-4ba3-9c40-2ba6a9c316ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.141152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dcc2c-f15a-4ba3-9c40-2ba6a9c316ab admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.163045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 788/1580] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:38 2022] POST /identity/v3/projects => generated 403 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:38.163926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a365bf4-9efc-4c96-80ec-25c0b30a3546 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.213256 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:38.224370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a365bf4-9efc-4c96-80ec-25c0b30a3546 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.229180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5211ccc-d8cd-4705-8e98-66cd4872741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.229837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5211ccc-d8cd-4705-8e98-66cd4872741a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.230119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5211ccc-d8cd-4705-8e98-66cd4872741a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.230387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 793/1581] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:38 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:38.230490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5211ccc-d8cd-4705-8e98-66cd4872741a admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.231110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5211ccc-d8cd-4705-8e98-66cd4872741a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.231305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dcc2c-f15a-4ba3-9c40-2ba6a9c316ab admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.232735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5211ccc-d8cd-4705-8e98-66cd4872741a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.241438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dcc2c-f15a-4ba3-9c40-2ba6a9c316ab admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.243598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 789/1582] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.244394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5211ccc-d8cd-4705-8e98-66cd4872741a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.247361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.249148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.249148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.249148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.249454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.251224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.256426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5211ccc-d8cd-4705-8e98-66cd4872741a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.261110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd5d2-3029-4338-af20-a048b6c4cfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.261747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd5d2-3029-4338-af20-a048b6c4cfc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.262027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd5d2-3029-4338-af20-a048b6c4cfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.262296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd5d2-3029-4338-af20-a048b6c4cfc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.263128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd5d2-3029-4338-af20-a048b6c4cfc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.264751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cd5d2-3029-4338-af20-a048b6c4cfc3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.275641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 790/1583] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.276423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd5d2-3029-4338-af20-a048b6c4cfc3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.285553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cd5d2-3029-4338-af20-a048b6c4cfc3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.290021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469de93-87ed-495c-a85f-fbd70c310422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.290662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469de93-87ed-495c-a85f-fbd70c310422 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.290939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469de93-87ed-495c-a85f-fbd70c310422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.291214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469de93-87ed-495c-a85f-fbd70c310422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.294741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469de93-87ed-495c-a85f-fbd70c310422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.294741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469de93-87ed-495c-a85f-fbd70c310422 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.304205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 791/1584] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.304945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469de93-87ed-495c-a85f-fbd70c310422 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.313979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469de93-87ed-495c-a85f-fbd70c310422 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.316860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 794/1585] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:38 2022] POST /identity/v3/users => generated 293 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:38.317676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.319852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f3089-95e8-4e6b-b10e-722da9c02c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.320568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f3089-95e8-4e6b-b10e-722da9c02c28 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.320850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f3089-95e8-4e6b-b10e-722da9c02c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.321014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:38.321118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f3089-95e8-4e6b-b10e-722da9c02c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.321822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f3089-95e8-4e6b-b10e-722da9c02c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.323544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f3089-95e8-4e6b-b10e-722da9c02c28 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.334346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 792/1586] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.335300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f3089-95e8-4e6b-b10e-722da9c02c28 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.344932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f3089-95e8-4e6b-b10e-722da9c02c28 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.349490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31332b6e-983b-4465-93c9-6ca7d5d76fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.350138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31332b6e-983b-4465-93c9-6ca7d5d76fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.350422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31332b6e-983b-4465-93c9-6ca7d5d76fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.350968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31332b6e-983b-4465-93c9-6ca7d5d76fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.351684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31332b6e-983b-4465-93c9-6ca7d5d76fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.353578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31332b6e-983b-4465-93c9-6ca7d5d76fa3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.380847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a6b5a-1bb2-415c-9f70-7026c959b7d5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.384576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa2bfe-5686-4c66-8f3b-703dcdeaeeae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.385222 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa2bfe-5686-4c66-8f3b-703dcdeaeeae service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.385507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa2bfe-5686-4c66-8f3b-703dcdeaeeae service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.385782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa2bfe-5686-4c66-8f3b-703dcdeaeeae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.394641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa2bfe-5686-4c66-8f3b-703dcdeaeeae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.396096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa2bfe-5686-4c66-8f3b-703dcdeaeeae service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.405356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 795/1587] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:38.406122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa2bfe-5686-4c66-8f3b-703dcdeaeeae service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.414730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa2bfe-5686-4c66-8f3b-703dcdeaeeae service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.419532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cbce0-21c8-4a26-b114-0bf773d01ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.420154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cbce0-21c8-4a26-b114-0bf773d01ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.420440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cbce0-21c8-4a26-b114-0bf773d01ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.420719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cbce0-21c8-4a26-b114-0bf773d01ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.421425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cbce0-21c8-4a26-b114-0bf773d01ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.423066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cbce0-21c8-4a26-b114-0bf773d01ea2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.424415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:INSERT=16 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:38.434341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 796/1588] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.439514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cbce0-21c8-4a26-b114-0bf773d01ea2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.439644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 793/1589] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:38 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:38.439644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31332b6e-983b-4465-93c9-6ca7d5d76fa3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.449796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cbce0-21c8-4a26-b114-0bf773d01ea2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.451365 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31332b6e-983b-4465-93c9-6ca7d5d76fa3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.454785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef26c8-6dd4-495a-859a-8faf6040960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.455489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef26c8-6dd4-495a-859a-8faf6040960b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.455772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef26c8-6dd4-495a-859a-8faf6040960b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.456047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef26c8-6dd4-495a-859a-8faf6040960b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.456771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef26c8-6dd4-495a-859a-8faf6040960b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.458439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef26c8-6dd4-495a-859a-8faf6040960b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.458634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c5824-6eef-4dca-8473-8b40fb43857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.459337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c5824-6eef-4dca-8473-8b40fb43857a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.459777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c5824-6eef-4dca-8473-8b40fb43857a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.459906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c5824-6eef-4dca-8473-8b40fb43857a admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.460621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c5824-6eef-4dca-8473-8b40fb43857a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.462210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c5824-6eef-4dca-8473-8b40fb43857a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.469454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 797/1590] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.470024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef26c8-6dd4-495a-859a-8faf6040960b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.475836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 794/1591] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.476898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c5824-6eef-4dca-8473-8b40fb43857a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.480010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef26c8-6dd4-495a-859a-8faf6040960b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.484532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ab1ba-7922-47e8-8823-3453d00831ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.485184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab1ba-7922-47e8-8823-3453d00831ac admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.485465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab1ba-7922-47e8-8823-3453d00831ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.485737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ab1ba-7922-47e8-8823-3453d00831ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.486552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ab1ba-7922-47e8-8823-3453d00831ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.486941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c5824-6eef-4dca-8473-8b40fb43857a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.488869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ab1ba-7922-47e8-8823-3453d00831ac admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.492175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7bdea-d2f1-4fc5-9d58-642388dd5b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.492832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7bdea-d2f1-4fc5-9d58-642388dd5b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.493144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7bdea-d2f1-4fc5-9d58-642388dd5b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.493541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7bdea-d2f1-4fc5-9d58-642388dd5b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.494261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7bdea-d2f1-4fc5-9d58-642388dd5b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.495878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7bdea-d2f1-4fc5-9d58-642388dd5b5a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.499048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 798/1592] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.501415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ab1ba-7922-47e8-8823-3453d00831ac admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.507276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 795/1593] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.507987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7bdea-d2f1-4fc5-9d58-642388dd5b5a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.513677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ab1ba-7922-47e8-8823-3453d00831ac admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.518514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7bdea-d2f1-4fc5-9d58-642388dd5b5a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.519313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa114dc-454e-4759-ab50-b08e6ca6b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.520024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa114dc-454e-4759-ab50-b08e6ca6b59b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.520325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa114dc-454e-4759-ab50-b08e6ca6b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.520630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa114dc-454e-4759-ab50-b08e6ca6b59b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.521359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa114dc-454e-4759-ab50-b08e6ca6b59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.522999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa114dc-454e-4759-ab50-b08e6ca6b59b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.523118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00342ec6-2951-4ee1-9fbf-921254ba5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.523714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00342ec6-2951-4ee1-9fbf-921254ba5c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.525594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00342ec6-2951-4ee1-9fbf-921254ba5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.525594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00342ec6-2951-4ee1-9fbf-921254ba5c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.525594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00342ec6-2951-4ee1-9fbf-921254ba5c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.526508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00342ec6-2951-4ee1-9fbf-921254ba5c13 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.533668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 799/1594] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.535094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa114dc-454e-4759-ab50-b08e6ca6b59b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.540541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 796/1595] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.541356 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00342ec6-2951-4ee1-9fbf-921254ba5c13 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.547158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa114dc-454e-4759-ab50-b08e6ca6b59b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.553606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de574c5d-bcf8-476c-805c-71a9086ca02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.559036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00342ec6-2951-4ee1-9fbf-921254ba5c13 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.559394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de574c5d-bcf8-476c-805c-71a9086ca02c admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.559394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de574c5d-bcf8-476c-805c-71a9086ca02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.559394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de574c5d-bcf8-476c-805c-71a9086ca02c admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.559394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de574c5d-bcf8-476c-805c-71a9086ca02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.559394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de574c5d-bcf8-476c-805c-71a9086ca02c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.559811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba780bf3-fa28-4402-aaf3-58f0198f2427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.560111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780bf3-fa28-4402-aaf3-58f0198f2427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.560447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780bf3-fa28-4402-aaf3-58f0198f2427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.560770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba780bf3-fa28-4402-aaf3-58f0198f2427 admin admin] PATH_INFO: `/v3/projects/877612289735450ea6bd527ebe7b81f3/users/5f9ae0e6a2534bb0b5d5e9327b01b238/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.570006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 800/1596] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.571314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de574c5d-bcf8-476c-805c-71a9086ca02c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.577033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba780bf3-fa28-4402-aaf3-58f0198f2427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=877612289735450ea6bd527ebe7b81f3, user_id=5f9ae0e6a2534bb0b5d5e9327b01b238, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.580864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba780bf3-fa28-4402-aaf3-58f0198f2427 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.583293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de574c5d-bcf8-476c-805c-71a9086ca02c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.588855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4250-facb-41a6-a751-bf2a92c72349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.588855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4250-facb-41a6-a751-bf2a92c72349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.589312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4250-facb-41a6-a751-bf2a92c72349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.589593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4250-facb-41a6-a751-bf2a92c72349 admin admin] PATH_INFO: `/v3/projects/1328a510264f4374baabad7b5243d8c2/users/cbcc15c786fb43669ac34a4897735c1d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.604786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e4250-facb-41a6-a751-bf2a92c72349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1328a510264f4374baabad7b5243d8c2, user_id=cbcc15c786fb43669ac34a4897735c1d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.607482 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e4250-facb-41a6-a751-bf2a92c72349 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.665548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 797/1597] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:38 2022] PUT /identity/v3/projects/877612289735450ea6bd527ebe7b81f3/users/5f9ae0e6a2534bb0b5d5e9327b01b238/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:38.666505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba780bf3-fa28-4402-aaf3-58f0198f2427 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.688930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 801/1598] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:38 2022] PUT /identity/v3/projects/1328a510264f4374baabad7b5243d8c2/users/cbcc15c786fb43669ac34a4897735c1d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:38.692406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4250-facb-41a6-a751-bf2a92c72349 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.735813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba780bf3-fa28-4402-aaf3-58f0198f2427 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.765046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4250-facb-41a6-a751-bf2a92c72349 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.770357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f4f75-c55d-4e4d-846f-6069fdc519a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.771036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f4f75-c55d-4e4d-846f-6069fdc519a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.775212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f4f75-c55d-4e4d-846f-6069fdc519a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.775212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f4f75-c55d-4e4d-846f-6069fdc519a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.775212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f4f75-c55d-4e4d-846f-6069fdc519a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.775212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f4f75-c55d-4e4d-846f-6069fdc519a9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.788117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 802/1599] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:38.789121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f4f75-c55d-4e4d-846f-6069fdc519a9 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.797185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:38.801496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f4f75-c55d-4e4d-846f-6069fdc519a9 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.807483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55df57-f1a2-4e33-a530-29bab09f32ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.814129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55df57-f1a2-4e33-a530-29bab09f32ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.814129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55df57-f1a2-4e33-a530-29bab09f32ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.814129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55df57-f1a2-4e33-a530-29bab09f32ca admin admin] PATH_INFO: `/v3/projects/553c41f9111e4be28dbb33b2d6d548bb/users/c1aec12109d74b929ddc89afb5c75040/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.817513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba9d73-e922-4050-bc1d-848982769cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:38.818412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba9d73-e922-4050-bc1d-848982769cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:38.818696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba9d73-e922-4050-bc1d-848982769cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:38.818998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba9d73-e922-4050-bc1d-848982769cbc admin admin] PATH_INFO: `/v3/projects/c1ac7800b9b34884bab3152f21212dc9/users/be1bf37435244cf8ba5e4b355fbff52f/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:38.829496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55df57-f1a2-4e33-a530-29bab09f32ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=553c41f9111e4be28dbb33b2d6d548bb, user_id=c1aec12109d74b929ddc89afb5c75040, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.834481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55df57-f1a2-4e33-a530-29bab09f32ca admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.835597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba9d73-e922-4050-bc1d-848982769cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ac7800b9b34884bab3152f21212dc9, user_id=be1bf37435244cf8ba5e4b355fbff52f, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:38.838633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba9d73-e922-4050-bc1d-848982769cbc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:38.921243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 798/1600] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:38 2022] PUT /identity/v3/projects/c1ac7800b9b34884bab3152f21212dc9/users/be1bf37435244cf8ba5e4b355fbff52f/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:38.921243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba9d73-e922-4050-bc1d-848982769cbc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.921563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 803/1601] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:38 2022] PUT /identity/v3/projects/553c41f9111e4be28dbb33b2d6d548bb/users/c1aec12109d74b929ddc89afb5c75040/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:38.921563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55df57-f1a2-4e33-a530-29bab09f32ca admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:38.993916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba9d73-e922-4050-bc1d-848982769cbc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:38.999239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a00ab8-fa74-46c2-bd64-9b996c07218b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.000667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a00ab8-fa74-46c2-bd64-9b996c07218b admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.000667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a00ab8-fa74-46c2-bd64-9b996c07218b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.000859 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a00ab8-fa74-46c2-bd64-9b996c07218b admin admin] PATH_INFO: `/v3/projects/5779b0ffa2244901a92adbee65b10421/users/15ff2cacb5c1450482597a2f5fa3c8a9/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.017071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a00ab8-fa74-46c2-bd64-9b996c07218b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5779b0ffa2244901a92adbee65b10421, user_id=15ff2cacb5c1450482597a2f5fa3c8a9, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.020296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a00ab8-fa74-46c2-bd64-9b996c07218b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.022031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55df57-f1a2-4e33-a530-29bab09f32ca admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.029468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758da65-8d6e-407f-904d-1f6c74d057a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.030139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758da65-8d6e-407f-904d-1f6c74d057a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.030424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758da65-8d6e-407f-904d-1f6c74d057a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.030700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758da65-8d6e-407f-904d-1f6c74d057a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.031461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758da65-8d6e-407f-904d-1f6c74d057a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.033055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758da65-8d6e-407f-904d-1f6c74d057a0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.034901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:39.044350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 804/1602] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:39.045580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b043c-1b6e-46a6-92e4-5ea4864ec106 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.045934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b043c-1b6e-46a6-92e4-5ea4864ec106 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.046211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b043c-1b6e-46a6-92e4-5ea4864ec106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.066163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ff0b043c-1b6e-46a6-92e4-5ea4864ec106 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:39.067554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ff0b043c-1b6e-46a6-92e4-5ea4864ec106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:39.075444 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 805/1603] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:39.076865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2a5e-e346-4d42-8e6f-1c7a4e55fb1a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.077160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2a5e-e346-4d42-8e6f-1c7a4e55fb1a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.077440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2a5e-e346-4d42-8e6f-1c7a4e55fb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.095121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4beb2a5e-e346-4d42-8e6f-1c7a4e55fb1a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:39.096381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4beb2a5e-e346-4d42-8e6f-1c7a4e55fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:39.104389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 806/1604] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:39.105176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb2a5e-e346-4d42-8e6f-1c7a4e55fb1a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:39.114052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 799/1605] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:38 2022] PUT /identity/v3/projects/5779b0ffa2244901a92adbee65b10421/users/15ff2cacb5c1450482597a2f5fa3c8a9/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:39.114943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929dac6-6809-4570-9075-c1d77f64aa77 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.119307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929dac6-6809-4570-9075-c1d77f64aa77 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.119307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929dac6-6809-4570-9075-c1d77f64aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.119715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb2a5e-e346-4d42-8e6f-1c7a4e55fb1a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.149088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-3929dac6-6809-4570-9075-c1d77f64aa77 None None] MFA Rules not processed for user `be1bf37435244cf8ba5e4b355fbff52f`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:39.150311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3929dac6-6809-4570-9075-c1d77f64aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:39.180759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997c2ac-60df-4807-a8ae-1d536916c45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.183532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997c2ac-60df-4807-a8ae-1d536916c45f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.183532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997c2ac-60df-4807-a8ae-1d536916c45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.183532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997c2ac-60df-4807-a8ae-1d536916c45f admin admin] PATH_INFO: `/v3/projects/876864a8f283481a81ee65a1c97d0f4f/users/993a59e8746948ab8a0f11d877314e02/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.200187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997c2ac-60df-4807-a8ae-1d536916c45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=876864a8f283481a81ee65a1c97d0f4f, user_id=993a59e8746948ab8a0f11d877314e02, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.202887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997c2ac-60df-4807-a8ae-1d536916c45f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.224607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 800/1606] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:39.225911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410f046-5b32-4b0c-82d8-22c742af905b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.226207 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410f046-5b32-4b0c-82d8-22c742af905b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.226541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410f046-5b32-4b0c-82d8-22c742af905b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.254715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8410f046-5b32-4b0c-82d8-22c742af905b None None] MFA Rules not processed for user `c1aec12109d74b929ddc89afb5c75040`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:39.255877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8410f046-5b32-4b0c-82d8-22c742af905b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:39.286588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:39.330254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 801/1607] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:39.331229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410f046-5b32-4b0c-82d8-22c742af905b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:39.344261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410f046-5b32-4b0c-82d8-22c742af905b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.346031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 807/1608] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:39 2022] PUT /identity/v3/projects/876864a8f283481a81ee65a1c97d0f4f/users/993a59e8746948ab8a0f11d877314e02/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:39.349936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997c2ac-60df-4807-a8ae-1d536916c45f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:39.414228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997c2ac-60df-4807-a8ae-1d536916c45f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.417813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edba7a-739d-4e9c-bf6c-0d39d9560438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.417984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8f58a-bb43-49fd-837a-3d6d542a3fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.418556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8f58a-bb43-49fd-837a-3d6d542a3fcc service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.418845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8f58a-bb43-49fd-837a-3d6d542a3fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.418947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edba7a-739d-4e9c-bf6c-0d39d9560438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.419129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8f58a-bb43-49fd-837a-3d6d542a3fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.419298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edba7a-739d-4e9c-bf6c-0d39d9560438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.419439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edba7a-739d-4e9c-bf6c-0d39d9560438 admin admin] PATH_INFO: `/v3/projects/8e7498feab634b50931bbadd2fa80827/users/85ad54b2480a4ad9b548deb6e2026d5b/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.429959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8f58a-bb43-49fd-837a-3d6d542a3fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.431507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8f58a-bb43-49fd-837a-3d6d542a3fcc service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.434832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edba7a-739d-4e9c-bf6c-0d39d9560438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7498feab634b50931bbadd2fa80827, user_id=85ad54b2480a4ad9b548deb6e2026d5b, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.437523 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edba7a-739d-4e9c-bf6c-0d39d9560438 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.441252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 808/1609] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:39.442038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8f58a-bb43-49fd-837a-3d6d542a3fcc service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:39.449570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8f58a-bb43-49fd-837a-3d6d542a3fcc service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.453181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffe736-0669-4adb-8561-8b6d8c0fa664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.453798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffe736-0669-4adb-8561-8b6d8c0fa664 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.454080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffe736-0669-4adb-8561-8b6d8c0fa664 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.454454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffe736-0669-4adb-8561-8b6d8c0fa664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.463570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffe736-0669-4adb-8561-8b6d8c0fa664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.465189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffe736-0669-4adb-8561-8b6d8c0fa664 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.476020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 809/1610] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:39.477370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d1c43-ce02-49a0-838c-95e99524f2f7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.477660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d1c43-ce02-49a0-838c-95e99524f2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.477934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d1c43-ce02-49a0-838c-95e99524f2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.495508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c22d1c43-ce02-49a0-838c-95e99524f2f7 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:39.496753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c22d1c43-ce02-49a0-838c-95e99524f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:39.504680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 810/1611] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:39.505488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d1c43-ce02-49a0-838c-95e99524f2f7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:39.515490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d1c43-ce02-49a0-838c-95e99524f2f7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.521421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352af61-f53a-4ccf-babc-59cdf94efaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.522539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352af61-f53a-4ccf-babc-59cdf94efaf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.522539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352af61-f53a-4ccf-babc-59cdf94efaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.522828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352af61-f53a-4ccf-babc-59cdf94efaf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.523690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352af61-f53a-4ccf-babc-59cdf94efaf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.525769 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352af61-f53a-4ccf-babc-59cdf94efaf2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.557355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 802/1612] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:39 2022] PUT /identity/v3/projects/8e7498feab634b50931bbadd2fa80827/users/85ad54b2480a4ad9b548deb6e2026d5b/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:39.559347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ae1fe-923e-4be5-ba5b-11def9f6f619 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.559611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ae1fe-923e-4be5-ba5b-11def9f6f619 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.559887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ae1fe-923e-4be5-ba5b-11def9f6f619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.576384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-243ae1fe-923e-4be5-ba5b-11def9f6f619 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:39.577588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-243ae1fe-923e-4be5-ba5b-11def9f6f619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:39.642946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 803/1613] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:39.648132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a78d-4648-424a-aeb5-f3d4eebf07a5 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.648434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a78d-4648-424a-aeb5-f3d4eebf07a5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.648703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271a78d-4648-424a-aeb5-f3d4eebf07a5 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.650832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 804/1614] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 11 11:25:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:25:39.651888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271a78d-4648-424a-aeb5-f3d4eebf07a5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:39.665434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 811/1615] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:39.666639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb7cf4-dbd6-4b8c-8c8a-319d6f8f44ad None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.666931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb7cf4-dbd6-4b8c-8c8a-319d6f8f44ad None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.667213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb7cf4-dbd6-4b8c-8c8a-319d6f8f44ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.704738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7beb7cf4-dbd6-4b8c-8c8a-319d6f8f44ad None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:39.705052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7beb7cf4-dbd6-4b8c-8c8a-319d6f8f44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:39.734832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:39.742933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271a78d-4648-424a-aeb5-f3d4eebf07a5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.775987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 812/1616] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:39.777962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0908bd2-439f-46a3-892d-23cc9511b3eb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.778258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0908bd2-439f-46a3-892d-23cc9511b3eb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.778540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0908bd2-439f-46a3-892d-23cc9511b3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.802346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:39.815476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea235fb4-1176-4202-89ea-1cdb243c8c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.817318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea235fb4-1176-4202-89ea-1cdb243c8c46 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.818570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea235fb4-1176-4202-89ea-1cdb243c8c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.819140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea235fb4-1176-4202-89ea-1cdb243c8c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.828154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e0908bd2-439f-46a3-892d-23cc9511b3eb None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:39.829516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e0908bd2-439f-46a3-892d-23cc9511b3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:39.833212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea235fb4-1176-4202-89ea-1cdb243c8c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.835051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea235fb4-1176-4202-89ea-1cdb243c8c46 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.851600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 805/1617] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:39.852538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea235fb4-1176-4202-89ea-1cdb243c8c46 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:39.894856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea235fb4-1176-4202-89ea-1cdb243c8c46 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.904664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb4f94-9c9e-4bd9-b259-5bcb91acb034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.905820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb4f94-9c9e-4bd9-b259-5bcb91acb034 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.906116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb4f94-9c9e-4bd9-b259-5bcb91acb034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.906780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb4f94-9c9e-4bd9-b259-5bcb91acb034 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.907575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb4f94-9c9e-4bd9-b259-5bcb91acb034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.910685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb4f94-9c9e-4bd9-b259-5bcb91acb034 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.940816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 813/1618] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:39.941828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0908bd2-439f-46a3-892d-23cc9511b3eb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:39.952348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0908bd2-439f-46a3-892d-23cc9511b3eb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:39.957382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1e82d-58f8-409c-b8ab-51b2a2a8ee88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:39.958143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1e82d-58f8-409c-b8ab-51b2a2a8ee88 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:39.958438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1e82d-58f8-409c-b8ab-51b2a2a8ee88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:39.958717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1e82d-58f8-409c-b8ab-51b2a2a8ee88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:39.969959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1e82d-58f8-409c-b8ab-51b2a2a8ee88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:39.974298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1e82d-58f8-409c-b8ab-51b2a2a8ee88 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:39.984451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 814/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:39.986356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1e82d-58f8-409c-b8ab-51b2a2a8ee88 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:40.001215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 806/1620] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:39 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:40.001215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb4f94-9c9e-4bd9-b259-5bcb91acb034 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:40.008813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1e82d-58f8-409c-b8ab-51b2a2a8ee88 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:40.013527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a9e62-9ea3-4313-93fa-caa4dff2057a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:40.016211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a9e62-9ea3-4313-93fa-caa4dff2057a service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.019086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a9e62-9ea3-4313-93fa-caa4dff2057a service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.019086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a9e62-9ea3-4313-93fa-caa4dff2057a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.023426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb4f94-9c9e-4bd9-b259-5bcb91acb034 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:40.034304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a9e62-9ea3-4313-93fa-caa4dff2057a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:40.036975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a9e62-9ea3-4313-93fa-caa4dff2057a service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:40.044732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516aa022-e1eb-4002-bb0a-80044a7c9b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:40.046374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516aa022-e1eb-4002-bb0a-80044a7c9b4b service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.046579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516aa022-e1eb-4002-bb0a-80044a7c9b4b service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.046921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516aa022-e1eb-4002-bb0a-80044a7c9b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.051172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 815/1621] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:40.054178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a9e62-9ea3-4313-93fa-caa4dff2057a service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:40.067604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a9e62-9ea3-4313-93fa-caa4dff2057a service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:40.086936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75832c-cbce-45e8-a40d-c46491f3bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:40.087615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75832c-cbce-45e8-a40d-c46491f3bd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.087907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75832c-cbce-45e8-a40d-c46491f3bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.088512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75832c-cbce-45e8-a40d-c46491f3bd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.088978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75832c-cbce-45e8-a40d-c46491f3bd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:40.090588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75832c-cbce-45e8-a40d-c46491f3bd5b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:40.107933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 816/1622] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:40.112027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75832c-cbce-45e8-a40d-c46491f3bd5b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:40.128338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75832c-cbce-45e8-a40d-c46491f3bd5b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:40.136016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae155f-692d-4db1-902e-1e7a33d0a304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:40.138139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae155f-692d-4db1-902e-1e7a33d0a304 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.139413 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae155f-692d-4db1-902e-1e7a33d0a304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.142053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae155f-692d-4db1-902e-1e7a33d0a304 admin admin] PATH_INFO: `/v3/projects/f56297e0ceb444f7bf40fc677187e5f8/users/c445d8358a6e4c1b86ab6372fcbd5eda/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.177664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae155f-692d-4db1-902e-1e7a33d0a304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56297e0ceb444f7bf40fc677187e5f8, user_id=c445d8358a6e4c1b86ab6372fcbd5eda, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:40.184993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae155f-692d-4db1-902e-1e7a33d0a304 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:40.194018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516aa022-e1eb-4002-bb0a-80044a7c9b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:40.197893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516aa022-e1eb-4002-bb0a-80044a7c9b4b service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:40.210262 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 807/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:40.246226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 817/1624] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:40 2022] PUT /identity/v3/projects/f56297e0ceb444f7bf40fc677187e5f8/users/c445d8358a6e4c1b86ab6372fcbd5eda/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:40.250260 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516aa022-e1eb-4002-bb0a-80044a7c9b4b service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:40.272971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516aa022-e1eb-4002-bb0a-80044a7c9b4b service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:40.371090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b90fd8-6f54-43c1-a15a-32ae839a1560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:40.371865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b90fd8-6f54-43c1-a15a-32ae839a1560 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.372176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b90fd8-6f54-43c1-a15a-32ae839a1560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.372456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b90fd8-6f54-43c1-a15a-32ae839a1560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.373217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b90fd8-6f54-43c1-a15a-32ae839a1560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:40.375219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b90fd8-6f54-43c1-a15a-32ae839a1560 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:40.388872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 808/1625] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:40.392875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae155f-692d-4db1-902e-1e7a33d0a304 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:40.407220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae155f-692d-4db1-902e-1e7a33d0a304 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:40.413318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab07aa-d6bb-4a7e-8187-fcfb8d0a1cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:40.414363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab07aa-d6bb-4a7e-8187-fcfb8d0a1cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.414655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab07aa-d6bb-4a7e-8187-fcfb8d0a1cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.415660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab07aa-d6bb-4a7e-8187-fcfb8d0a1cbf admin admin] PATH_INFO: `/v3/projects/f56297e0ceb444f7bf40fc677187e5f8/users/c445d8358a6e4c1b86ab6372fcbd5eda/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.417852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab07aa-d6bb-4a7e-8187-fcfb8d0a1cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56297e0ceb444f7bf40fc677187e5f8, user_id=c445d8358a6e4c1b86ab6372fcbd5eda, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:40.420732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab07aa-d6bb-4a7e-8187-fcfb8d0a1cbf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:40.456717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 818/1626] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:40 2022] PUT /identity/v3/projects/f56297e0ceb444f7bf40fc677187e5f8/users/c445d8358a6e4c1b86ab6372fcbd5eda/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:40.468042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3fd49-5bf8-4e76-935e-133239d00d15 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.468339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3fd49-5bf8-4e76-935e-133239d00d15 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.468687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3fd49-5bf8-4e76-935e-133239d00d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.505345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-06a3fd49-5bf8-4e76-935e-133239d00d15 None None] MFA Rules not processed for user `c445d8358a6e4c1b86ab6372fcbd5eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:40.506713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-06a3fd49-5bf8-4e76-935e-133239d00d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:40.534910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:40.582626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe4b13-d764-4d8f-85b4-be59ea204bf0 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.583628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe4b13-d764-4d8f-85b4-be59ea204bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.587359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe4b13-d764-4d8f-85b4-be59ea204bf0 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.587359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 819/1627] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 11 11:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:25:40.593664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38279d82-6dd9-4aab-bdc4-a28e6d266217 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.594156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38279d82-6dd9-4aab-bdc4-a28e6d266217 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.594683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38279d82-6dd9-4aab-bdc4-a28e6d266217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.597880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 809/1628] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:40 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:40.612258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3fd49-5bf8-4e76-935e-133239d00d15 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:40.618568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-38279d82-6dd9-4aab-bdc4-a28e6d266217 None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:40.620045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-38279d82-6dd9-4aab-bdc4-a28e6d266217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:40.658485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:40.713777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3fd49-5bf8-4e76-935e-133239d00d15 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:40.724221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff1cf1-29fc-47d2-ab3d-ed1a3ef80bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:40.725692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff1cf1-29fc-47d2-ab3d-ed1a3ef80bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.725974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff1cf1-29fc-47d2-ab3d-ed1a3ef80bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.727029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff1cf1-29fc-47d2-ab3d-ed1a3ef80bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.756923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff1cf1-29fc-47d2-ab3d-ed1a3ef80bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:40.758778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff1cf1-29fc-47d2-ab3d-ed1a3ef80bd0 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:40.784823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 820/1629] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:25:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:40.788946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 810/1630] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:40 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:25:40.802276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38279d82-6dd9-4aab-bdc4-a28e6d266217 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:40.823868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38279d82-6dd9-4aab-bdc4-a28e6d266217 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:40.831853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce662fe0-cfc0-4849-83bc-f180a28919e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:40.832096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce662fe0-cfc0-4849-83bc-f180a28919e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:40.832516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce662fe0-cfc0-4849-83bc-f180a28919e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:40.832950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce662fe0-cfc0-4849-83bc-f180a28919e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:40.859539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce662fe0-cfc0-4849-83bc-f180a28919e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:40.859539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce662fe0-cfc0-4849-83bc-f180a28919e7 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:40.884530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 821/1631] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:42.615440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada2584-83cf-4f35-a1e7-6c2d7cf91984 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:42.615982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada2584-83cf-4f35-a1e7-6c2d7cf91984 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:42.615982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada2584-83cf-4f35-a1e7-6c2d7cf91984 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:42.617633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 811/1632] 10.222.0.26 () {54 vars in 961 bytes} [Mon Jul 11 11:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:25:42.626071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc28668-dc01-499e-9e2b-c5182d71a4d3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:42.626644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc28668-dc01-499e-9e2b-c5182d71a4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:42.626644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc28668-dc01-499e-9e2b-c5182d71a4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:42.654211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0cc28668-dc01-499e-9e2b-c5182d71a4d3 None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:42.656249 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0cc28668-dc01-499e-9e2b-c5182d71a4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:42.665493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 822/1633] 10.222.0.26 () {58 vars in 1058 bytes} [Mon Jul 11 11:25:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:42.676440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd846d-4573-44ec-9698-4665a46c6c42 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:42.676977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd846d-4573-44ec-9698-4665a46c6c42 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:42.677264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd846d-4573-44ec-9698-4665a46c6c42 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:42.678889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 812/1634] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 11 11:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:25:42.685057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2a897-fae0-483f-bf21-27a117117bca None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:42.685355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2a897-fae0-483f-bf21-27a117117bca None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:42.685906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2a897-fae0-483f-bf21-27a117117bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:42.726375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-94d2a897-fae0-483f-bf21-27a117117bca None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:42.728359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-94d2a897-fae0-483f-bf21-27a117117bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:42.770119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 823/1635] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:25:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:42.791992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd846d-4573-44ec-9698-4665a46c6c42 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:42.827456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd846d-4573-44ec-9698-4665a46c6c42 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:42.834813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69051d8-d90e-4b6a-95d1-b3adb997bb1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:42.835556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69051d8-d90e-4b6a-95d1-b3adb997bb1d service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:42.835838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69051d8-d90e-4b6a-95d1-b3adb997bb1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:42.836113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69051d8-d90e-4b6a-95d1-b3adb997bb1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:42.851645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69051d8-d90e-4b6a-95d1-b3adb997bb1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:42.854420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69051d8-d90e-4b6a-95d1-b3adb997bb1d service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:42.875073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 813/1636] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:43.249573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f22b7-e518-4d93-b004-b3cdf97911a1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:43.250370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f22b7-e518-4d93-b004-b3cdf97911a1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:43.250881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f22b7-e518-4d93-b004-b3cdf97911a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:43.302910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-483f22b7-e518-4d93-b004-b3cdf97911a1 None None] MFA Rules not processed for user `85ad54b2480a4ad9b548deb6e2026d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:43.304903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-483f22b7-e518-4d93-b004-b3cdf97911a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:43.403381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 824/1637] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:43.406771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69051d8-d90e-4b6a-95d1-b3adb997bb1d service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:43.633735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69051d8-d90e-4b6a-95d1-b3adb997bb1d service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:43.638692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ec51f-9636-4330-91d1-9496b8c76054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:43.639521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec51f-9636-4330-91d1-9496b8c76054 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:43.640040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec51f-9636-4330-91d1-9496b8c76054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:43.640525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ec51f-9636-4330-91d1-9496b8c76054 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:43.641419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ec51f-9636-4330-91d1-9496b8c76054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:43.644391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ec51f-9636-4330-91d1-9496b8c76054 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:43.939142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 814/1638] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:43 2022] POST /identity/v3/projects => generated 403 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:43.944574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f22b7-e518-4d93-b004-b3cdf97911a1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:43.980924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f22b7-e518-4d93-b004-b3cdf97911a1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:44.121029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dd23f-21e9-4e66-a941-131dfbe26b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:44.121729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd23f-21e9-4e66-a941-131dfbe26b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:44.122037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd23f-21e9-4e66-a941-131dfbe26b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:44.122301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686dd23f-21e9-4e66-a941-131dfbe26b58 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:44.123010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686dd23f-21e9-4e66-a941-131dfbe26b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:44.124996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686dd23f-21e9-4e66-a941-131dfbe26b58 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:44.282522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 825/1639] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:43 2022] POST /identity/v3/users => generated 308 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:44.286398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ec51f-9636-4330-91d1-9496b8c76054 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:44.308184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ec51f-9636-4330-91d1-9496b8c76054 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:44.323977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b3cff-348f-447c-8ceb-9298594bc681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:44.324679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b3cff-348f-447c-8ceb-9298594bc681 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:44.324968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b3cff-348f-447c-8ceb-9298594bc681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:44.325248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b3cff-348f-447c-8ceb-9298594bc681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:44.326055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b3cff-348f-447c-8ceb-9298594bc681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:44.332493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b3cff-348f-447c-8ceb-9298594bc681 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:44.360205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 815/1640] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:44.362385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dd23f-21e9-4e66-a941-131dfbe26b58 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:44.375728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686dd23f-21e9-4e66-a941-131dfbe26b58 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:44.391014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e1ee6-5c42-4f26-a14a-893d5ac50625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:44.397031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1ee6-5c42-4f26-a14a-893d5ac50625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:44.397031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1ee6-5c42-4f26-a14a-893d5ac50625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:44.397031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e1ee6-5c42-4f26-a14a-893d5ac50625 admin admin] PATH_INFO: `/v3/projects/cfba21ecf40a42d08cb398fad71783f6/users/1df400ae412245b3854350028b055794/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:44.422503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:44.430291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e1ee6-5c42-4f26-a14a-893d5ac50625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfba21ecf40a42d08cb398fad71783f6, user_id=1df400ae412245b3854350028b055794, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:44.430291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e1ee6-5c42-4f26-a14a-893d5ac50625 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:44.525170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 826/1641] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:44 2022] PUT /identity/v3/projects/cfba21ecf40a42d08cb398fad71783f6/users/1df400ae412245b3854350028b055794/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:44.527172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b3cff-348f-447c-8ceb-9298594bc681 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:44.543735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b3cff-348f-447c-8ceb-9298594bc681 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:44.580536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:44.697238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327183b-e3a8-4d08-9f5b-fa25dc32c066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:44.698875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327183b-e3a8-4d08-9f5b-fa25dc32c066 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:44.699489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327183b-e3a8-4d08-9f5b-fa25dc32c066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:44.699977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327183b-e3a8-4d08-9f5b-fa25dc32c066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:44.700916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327183b-e3a8-4d08-9f5b-fa25dc32c066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:44.702917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327183b-e3a8-4d08-9f5b-fa25dc32c066 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:44.739120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 816/1642] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:44.741464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e1ee6-5c42-4f26-a14a-893d5ac50625 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:44.761976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e1ee6-5c42-4f26-a14a-893d5ac50625 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:44.773023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97da06f-416b-4d05-b347-a5a5120c7017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:44.773940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da06f-416b-4d05-b347-a5a5120c7017 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:44.774229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da06f-416b-4d05-b347-a5a5120c7017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:44.774506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da06f-416b-4d05-b347-a5a5120c7017 admin admin] PATH_INFO: `/v3/projects/cfba21ecf40a42d08cb398fad71783f6/users/1df400ae412245b3854350028b055794/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:44.784620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97da06f-416b-4d05-b347-a5a5120c7017 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfba21ecf40a42d08cb398fad71783f6, user_id=1df400ae412245b3854350028b055794, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:44.789147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97da06f-416b-4d05-b347-a5a5120c7017 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:44.812647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 827/1643] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:44 2022] PUT /identity/v3/projects/cfba21ecf40a42d08cb398fad71783f6/users/1df400ae412245b3854350028b055794/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:45.482669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327183b-e3a8-4d08-9f5b-fa25dc32c066 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:45.496638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327183b-e3a8-4d08-9f5b-fa25dc32c066 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:45.569580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:45.570776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:45.572266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:45.572880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] PATH_INFO: `/v3/users/be1bf37435244cf8ba5e4b355fbff52f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:45.574844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1bf37435244cf8ba5e4b355fbff52f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:45.576435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: WARNING py.warnings [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:25:45.576435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: warnings.warn(msg) Jul 11 11:25:45.576435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: Jul 11 11:25:45.578285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:45.829248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1bf37435244cf8ba5e4b355fbff52f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:45.830573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1bf37435244cf8ba5e4b355fbff52f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:45.832042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1bf37435244cf8ba5e4b355fbff52f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:45.843551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1bf37435244cf8ba5e4b355fbff52f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:45.868690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1bf37435244cf8ba5e4b355fbff52f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:46.018644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 817/1644] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:25:45 2022] DELETE /identity/v3/users/be1bf37435244cf8ba5e4b355fbff52f => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:25:46.020616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b4d2-8c5e-4ff0-afc0-841c4a464078 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:46.021117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b4d2-8c5e-4ff0-afc0-841c4a464078 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:46.021234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b4d2-8c5e-4ff0-afc0-841c4a464078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:46.045481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c667b4d2-8c5e-4ff0-afc0-841c4a464078 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:46.045481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c667b4d2-8c5e-4ff0-afc0-841c4a464078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:46.127163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 828/1645] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:46.138155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:46.139933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:46.183764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:46.214931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fcf0e-77d3-4018-90ba-8f06031ac0c9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:46.220997 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35283cbf-44d9-4b0a-8bf7-ec522555fd4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:46.222389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35283cbf-44d9-4b0a-8bf7-ec522555fd4a service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:46.227461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35283cbf-44d9-4b0a-8bf7-ec522555fd4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:46.227981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35283cbf-44d9-4b0a-8bf7-ec522555fd4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:46.241780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35283cbf-44d9-4b0a-8bf7-ec522555fd4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:46.243171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35283cbf-44d9-4b0a-8bf7-ec522555fd4a service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:46.269463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 818/1646] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:47.829561 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b4d2-8c5e-4ff0-afc0-841c4a464078 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:47.831177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c667b4d2-8c5e-4ff0-afc0-841c4a464078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:47.859875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b4d2-8c5e-4ff0-afc0-841c4a464078 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:47.872198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:47.873198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:47.873655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:47.874032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] PATH_INFO: `/v3/projects/c1ac7800b9b34884bab3152f21212dc9` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:47.875937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ac7800b9b34884bab3152f21212dc9)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:47.879567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5453c7-e558-4a3c-b6e1-f43ce8daa603 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:47.880062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5453c7-e558-4a3c-b6e1-f43ce8daa603 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:47.880302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5453c7-e558-4a3c-b6e1-f43ce8daa603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:47.892225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: WARNING py.warnings [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 11:25:47.892225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: warnings.warn(msg) Jul 11 11:25:47.892225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: Jul 11 11:25:47.901834 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:47.962883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-db5453c7-e558-4a3c-b6e1-f43ce8daa603 None None] MFA Rules not processed for user `1df400ae412245b3854350028b055794`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:47.966389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-db5453c7-e558-4a3c-b6e1-f43ce8daa603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:48.144007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ac7800b9b34884bab3152f21212dc9', 'request_id': 'req-700896da-a1fa-48ab-a314-e8ff403cae49'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.157408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 819/1647] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:47 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:48.182281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ac7800b9b34884bab3152f21212dc9', 'request_id': 'req-700896da-a1fa-48ab-a314-e8ff403cae49'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.293141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ac7800b9b34884bab3152f21212dc9', 'request_id': 'req-700896da-a1fa-48ab-a314-e8ff403cae49'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.333373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] The token cache is being invalidate because project c1ac7800b9b34884bab3152f21212dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:25:48.337994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.339006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.340825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 829/1648] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:25:47 2022] DELETE /identity/v3/projects/c1ac7800b9b34884bab3152f21212dc9 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:25:48.346445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5453c7-e558-4a3c-b6e1-f43ce8daa603 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:48.365655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5453c7-e558-4a3c-b6e1-f43ce8daa603 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:48.495062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:48.496203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:48.498112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:48.498112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] PATH_INFO: `/v3/users/c445d8358a6e4c1b86ab6372fcbd5eda` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:48.499963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c445d8358a6e4c1b86ab6372fcbd5eda)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:48.506529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:48.795051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c445d8358a6e4c1b86ab6372fcbd5eda'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.796894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c445d8358a6e4c1b86ab6372fcbd5eda'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.797881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c445d8358a6e4c1b86ab6372fcbd5eda'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.816099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c445d8358a6e4c1b86ab6372fcbd5eda'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.844405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:48.862477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-82efd428-6aab-4802-8c99-344f7619beb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c445d8358a6e4c1b86ab6372fcbd5eda'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:48.951104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 820/1649] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:25:48 2022] DELETE /identity/v3/users/c445d8358a6e4c1b86ab6372fcbd5eda => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:51.243157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:51.244831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:51.273104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700896da-a1fa-48ab-a314-e8ff403cae49 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:51.431655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:51.435283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:51.435728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:51.442089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] PATH_INFO: `/v3/projects/f56297e0ceb444f7bf40fc677187e5f8` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:51.442089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f56297e0ceb444f7bf40fc677187e5f8)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:51.442089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:51.454491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:51.557384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f78ca-92a2-43c7-bf45-6625f0fae718 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:51.557948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f78ca-92a2-43c7-bf45-6625f0fae718 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:51.557948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f78ca-92a2-43c7-bf45-6625f0fae718 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:51.559392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 821/1650] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:25:51.572854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eff9f9-c4a0-4961-8afa-07dce32200ec None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:51.573405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eff9f9-c4a0-4961-8afa-07dce32200ec None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:51.573693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eff9f9-c4a0-4961-8afa-07dce32200ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:51.624720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-09eff9f9-c4a0-4961-8afa-07dce32200ec None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:51.631809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-09eff9f9-c4a0-4961-8afa-07dce32200ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:51.676499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f56297e0ceb444f7bf40fc677187e5f8', 'request_id': 'req-da39846f-bef0-44d4-a371-ec3a7eef6d06'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:51.686548 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f56297e0ceb444f7bf40fc677187e5f8', 'request_id': 'req-da39846f-bef0-44d4-a371-ec3a7eef6d06'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:51.777765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 822/1651] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:51.778967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eff9f9-c4a0-4961-8afa-07dce32200ec None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:51.790547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-09eff9f9-c4a0-4961-8afa-07dce32200ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:51.808251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f56297e0ceb444f7bf40fc677187e5f8', 'request_id': 'req-da39846f-bef0-44d4-a371-ec3a7eef6d06'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:51.810704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eff9f9-c4a0-4961-8afa-07dce32200ec None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:51.860515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] The token cache is being invalidate because project f56297e0ceb444f7bf40fc677187e5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:25:51.869999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:51.875685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-da39846f-bef0-44d4-a371-ec3a7eef6d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:51.893702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 830/1652] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:25:51 2022] DELETE /identity/v3/projects/f56297e0ceb444f7bf40fc677187e5f8 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:51.901239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79d9b-4411-4913-97a8-6441595da280 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:51.901537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79d9b-4411-4913-97a8-6441595da280 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:51.905241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79d9b-4411-4913-97a8-6441595da280 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:51.916157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 831/1653] 10.222.0.26 () {54 vars in 960 bytes} [Mon Jul 11 11:25:51 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:25:51.921177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4227b-8d9e-4f72-8533-7bb034c3fa24 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:51.921448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4227b-8d9e-4f72-8533-7bb034c3fa24 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:51.921715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4227b-8d9e-4f72-8533-7bb034c3fa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:51.942981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:51.945565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:51.946138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:51.947280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] PATH_INFO: `/v3/users/c1aec12109d74b929ddc89afb5c75040` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:51.950316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1aec12109d74b929ddc89afb5c75040)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:51.956016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:51.991480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1da4227b-8d9e-4f72-8533-7bb034c3fa24 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:51.996119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1da4227b-8d9e-4f72-8533-7bb034c3fa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:52.031106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 832/1654] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:52.041789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158af3-6fc3-4bf3-8aac-d44c6aab98e6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:52.045547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158af3-6fc3-4bf3-8aac-d44c6aab98e6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:52.046028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af158af3-6fc3-4bf3-8aac-d44c6aab98e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:52.122831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-af158af3-6fc3-4bf3-8aac-d44c6aab98e6 None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:52.128482 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-af158af3-6fc3-4bf3-8aac-d44c6aab98e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:52.241807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1aec12109d74b929ddc89afb5c75040'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:52.242609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1aec12109d74b929ddc89afb5c75040'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:52.243341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1aec12109d74b929ddc89afb5c75040'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:52.251101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1aec12109d74b929ddc89afb5c75040'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:52.255630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 833/1655] 10.222.0.26 () {58 vars in 1057 bytes} [Mon Jul 11 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:52.259725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fae79a-a04f-4932-a8bc-12207879536f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:52.264168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fae79a-a04f-4932-a8bc-12207879536f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:52.264168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fae79a-a04f-4932-a8bc-12207879536f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:52.266527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1aec12109d74b929ddc89afb5c75040'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:52.293015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b5fae79a-a04f-4932-a8bc-12207879536f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:52.294733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b5fae79a-a04f-4932-a8bc-12207879536f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:52.304419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 834/1656] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:52.305415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fae79a-a04f-4932-a8bc-12207879536f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:52.323590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fae79a-a04f-4932-a8bc-12207879536f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:52.332037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1a014-5350-4fc3-81e0-1d13d6192842 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:52.332977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1a014-5350-4fc3-81e0-1d13d6192842 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:52.333473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1a014-5350-4fc3-81e0-1d13d6192842 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:52.333965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1a014-5350-4fc3-81e0-1d13d6192842 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:52.337549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-aaf1a014-5350-4fc3-81e0-1d13d6192842 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:52.353312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1a014-5350-4fc3-81e0-1d13d6192842 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:52.355677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1a014-5350-4fc3-81e0-1d13d6192842 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:52.363308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:52.369472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 823/1657] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:25:51 2022] DELETE /identity/v3/users/c1aec12109d74b929ddc89afb5c75040 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:52.370568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:52.388358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 835/1658] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:52.391332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac8f50-c4b5-4d66-85c9-01c4882c7a6d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:52.391615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac8f50-c4b5-4d66-85c9-01c4882c7a6d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:52.391889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac8f50-c4b5-4d66-85c9-01c4882c7a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:52.438764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2cac8f50-c4b5-4d66-85c9-01c4882c7a6d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:52.440110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2cac8f50-c4b5-4d66-85c9-01c4882c7a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:52.582958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e93ef-6549-4050-8f45-4f7d9ca7e7b6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:52.594876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:52.595597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:52.600757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:52.601124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:52.603056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:52.604164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:52.605819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:52.633183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 836/1659] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:52.640003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 824/1660] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:52.648806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac8f50-c4b5-4d66-85c9-01c4882c7a6d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:52.655782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:52.657959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:52.672730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac8f50-c4b5-4d66-85c9-01c4882c7a6d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:52.683990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc3dea-9ca6-481c-b169-12d2234b63bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:52.684962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3dea-9ca6-481c-b169-12d2234b63bb admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:52.685261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3dea-9ca6-481c-b169-12d2234b63bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:52.685557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc3dea-9ca6-481c-b169-12d2234b63bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:52.694653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc3dea-9ca6-481c-b169-12d2234b63bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:52.698174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc3dea-9ca6-481c-b169-12d2234b63bb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:52.809154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0727c8-2262-45ca-a6c8-10ff5d779a54 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:52.817655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c19ac8-7efe-4139-855b-de349e7ea66c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:52.835692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c19ac8-7efe-4139-855b-de349e7ea66c service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:52.835692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c19ac8-7efe-4139-855b-de349e7ea66c service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:52.835692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c19ac8-7efe-4139-855b-de349e7ea66c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:52.883096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 837/1661] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:52 2022] POST /identity/v3/projects => generated 387 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:52.887342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43ab36-c480-4cc4-b2b0-cc7f4f8d3a5d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:52.887342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43ab36-c480-4cc4-b2b0-cc7f4f8d3a5d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:52.889638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43ab36-c480-4cc4-b2b0-cc7f4f8d3a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:52.894134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c19ac8-7efe-4139-855b-de349e7ea66c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:52.898530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c19ac8-7efe-4139-855b-de349e7ea66c service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:52.937780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 825/1662] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:52.938659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c19ac8-7efe-4139-855b-de349e7ea66c service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:52.973363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c19ac8-7efe-4139-855b-de349e7ea66c service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:52.977273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7e43ab36-c480-4cc4-b2b0-cc7f4f8d3a5d None None] MFA Rules not processed for user `e47c365946cf4264a2d2205ace04379d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:52.979806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7e43ab36-c480-4cc4-b2b0-cc7f4f8d3a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:53.043229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:53.152502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c87a-4d2c-4bd0-af8d-9ab51c6a1eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:53.153073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c87a-4d2c-4bd0-af8d-9ab51c6a1eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:53.154568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c87a-4d2c-4bd0-af8d-9ab51c6a1eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:53.154867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c87a-4d2c-4bd0-af8d-9ab51c6a1eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:53.156246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149c87a-4d2c-4bd0-af8d-9ab51c6a1eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:53.160474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149c87a-4d2c-4bd0-af8d-9ab51c6a1eaa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:53.199806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 838/1663] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:53.201447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5c7c-58db-4b9e-a213-0a153cd137ff None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:53.201751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5c7c-58db-4b9e-a213-0a153cd137ff None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:53.202088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c5c7c-58db-4b9e-a213-0a153cd137ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:53.248021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-394c5c7c-58db-4b9e-a213-0a153cd137ff None None] MFA Rules not processed for user `e47c365946cf4264a2d2205ace04379d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:53.251618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-394c5c7c-58db-4b9e-a213-0a153cd137ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:53.273633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 839/1664] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:53.285828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c5c7c-58db-4b9e-a213-0a153cd137ff None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:53.287519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-394c5c7c-58db-4b9e-a213-0a153cd137ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:53.343093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 826/1665] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:52 2022] POST /identity/v3/users => generated 294 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:53.344474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c87a-4d2c-4bd0-af8d-9ab51c6a1eaa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:53.371485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c87a-4d2c-4bd0-af8d-9ab51c6a1eaa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:53.389917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a86bf-cb66-4ff7-a234-36e926b55159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:53.394015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a86bf-cb66-4ff7-a234-36e926b55159 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:53.394015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a86bf-cb66-4ff7-a234-36e926b55159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:53.394015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a86bf-cb66-4ff7-a234-36e926b55159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:53.394015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a86bf-cb66-4ff7-a234-36e926b55159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:53.396253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a86bf-cb66-4ff7-a234-36e926b55159 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:53.424655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 827/1666] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:53.430615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a86bf-cb66-4ff7-a234-36e926b55159 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:53.457528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a86bf-cb66-4ff7-a234-36e926b55159 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:53.468300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c5c7c-58db-4b9e-a213-0a153cd137ff None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:53.470245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517cbf3-2413-443c-9b30-7594330d1cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:53.471879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517cbf3-2413-443c-9b30-7594330d1cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:53.472273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517cbf3-2413-443c-9b30-7594330d1cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:53.472674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517cbf3-2413-443c-9b30-7594330d1cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:53.473539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517cbf3-2413-443c-9b30-7594330d1cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:53.475436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517cbf3-2413-443c-9b30-7594330d1cf2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:53.495617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a41aa-a4bf-4116-a236-184253aa6ff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:53.502899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a41aa-a4bf-4116-a236-184253aa6ff9 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:53.503371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 828/1667] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:53.503498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a41aa-a4bf-4116-a236-184253aa6ff9 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:53.503498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a41aa-a4bf-4116-a236-184253aa6ff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:53.504530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517cbf3-2413-443c-9b30-7594330d1cf2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:53.547056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a41aa-a4bf-4116-a236-184253aa6ff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:53.553998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a41aa-a4bf-4116-a236-184253aa6ff9 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:53.557080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517cbf3-2413-443c-9b30-7594330d1cf2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:53.567354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd83094-7ad1-42c5-afc5-1b59c604a3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:53.568992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd83094-7ad1-42c5-afc5-1b59c604a3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:53.569508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd83094-7ad1-42c5-afc5-1b59c604a3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:53.570047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd83094-7ad1-42c5-afc5-1b59c604a3dc admin admin] PATH_INFO: `/v3/projects/b438e3f541c94c67a5ef6a7d43aa3fb2/users/8600b1c9f8744f9da418d06e9fcf5d90/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:53.581532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 840/1668] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:53.602656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd83094-7ad1-42c5-afc5-1b59c604a3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b438e3f541c94c67a5ef6a7d43aa3fb2, user_id=8600b1c9f8744f9da418d06e9fcf5d90, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:53.607217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd83094-7ad1-42c5-afc5-1b59c604a3dc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:53.760069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f62e-e4cb-464b-8c2a-942696611026 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:53.760361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f62e-e4cb-464b-8c2a-942696611026 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:53.760640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f62e-e4cb-464b-8c2a-942696611026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:53.761491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 829/1669] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:53 2022] PUT /identity/v3/projects/b438e3f541c94c67a5ef6a7d43aa3fb2/users/8600b1c9f8744f9da418d06e9fcf5d90/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:53.807120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8e56f62e-e4cb-464b-8c2a-942696611026 None None] MFA Rules not processed for user `8600b1c9f8744f9da418d06e9fcf5d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:53.808402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8e56f62e-e4cb-464b-8c2a-942696611026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:53.984683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 841/1670] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:53.989941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd83094-7ad1-42c5-afc5-1b59c604a3dc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:54.249641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd83094-7ad1-42c5-afc5-1b59c604a3dc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:54.268856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9623cf3-1012-426e-978c-c4eecb124059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:54.274118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623cf3-1012-426e-978c-c4eecb124059 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:54.274118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623cf3-1012-426e-978c-c4eecb124059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:54.274118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9623cf3-1012-426e-978c-c4eecb124059 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:54.274118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9623cf3-1012-426e-978c-c4eecb124059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:54.274118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9623cf3-1012-426e-978c-c4eecb124059 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:54.423016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 830/1671] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:53 2022] POST /identity/v3/projects => generated 389 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:54.424980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56f62e-e4cb-464b-8c2a-942696611026 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:54.442415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56f62e-e4cb-464b-8c2a-942696611026 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:54.585389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:54.610286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:54.612198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:54.612506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:54.612785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:54.614845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:54.623201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:54.759622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 842/1672] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:54 2022] POST /identity/v3/users => generated 301 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:54.763756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9623cf3-1012-426e-978c-c4eecb124059 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:54.766265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:54.776656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:54.785028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9623cf3-1012-426e-978c-c4eecb124059 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:54.806507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f1c20-56ec-4729-9c31-eba400d275d5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:54.811808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066620dc-7d2c-4e26-a9f3-e7d2e9d108a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:54.812503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066620dc-7d2c-4e26-a9f3-e7d2e9d108a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:54.812794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066620dc-7d2c-4e26-a9f3-e7d2e9d108a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:54.813072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066620dc-7d2c-4e26-a9f3-e7d2e9d108a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:54.814345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066620dc-7d2c-4e26-a9f3-e7d2e9d108a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:54.816705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066620dc-7d2c-4e26-a9f3-e7d2e9d108a2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:54.833612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:54.834531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:54.834815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:54.835095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] PATH_INFO: `/v3/users/85ad54b2480a4ad9b548deb6e2026d5b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:54.837361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85ad54b2480a4ad9b548deb6e2026d5b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:54.839502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:54.860627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 831/1673] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:54.862686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066620dc-7d2c-4e26-a9f3-e7d2e9d108a2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:54.881716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066620dc-7d2c-4e26-a9f3-e7d2e9d108a2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:54.892445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8370c-46fc-4301-8b47-7118ace94b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:54.893739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8370c-46fc-4301-8b47-7118ace94b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:54.893739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8370c-46fc-4301-8b47-7118ace94b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:54.893930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8370c-46fc-4301-8b47-7118ace94b9a admin admin] PATH_INFO: `/v3/projects/4d9bcbb76be44039a6ebae8686f43b48/users/74d5ece9abcb4b7ea54ca756523080d0/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:54.933804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8370c-46fc-4301-8b47-7118ace94b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9bcbb76be44039a6ebae8686f43b48, user_id=74d5ece9abcb4b7ea54ca756523080d0, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:54.936641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8370c-46fc-4301-8b47-7118ace94b9a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:55.052265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 832/1674] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:54 2022] PUT /identity/v3/projects/4d9bcbb76be44039a6ebae8686f43b48/users/74d5ece9abcb4b7ea54ca756523080d0/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:25:55.060974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8370c-46fc-4301-8b47-7118ace94b9a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:55.147816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85ad54b2480a4ad9b548deb6e2026d5b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:55.152257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85ad54b2480a4ad9b548deb6e2026d5b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:55.152257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85ad54b2480a4ad9b548deb6e2026d5b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:55.168692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85ad54b2480a4ad9b548deb6e2026d5b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:55.193984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-bf42541d-d5e3-465f-8805-23d5a0c30a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85ad54b2480a4ad9b548deb6e2026d5b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:55.221480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8370c-46fc-4301-8b47-7118ace94b9a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:55.228008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:55.231246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:55.231574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:55.231863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:55.234554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:55.236798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:55.257210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 833/1675] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:55.258684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:55.264322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:55.289313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400d6cf-97eb-4942-bd2c-0aa02bba69e6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:55.296267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb5d35-3091-43e4-b67f-b7119cfe177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:55.297852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5d35-3091-43e4-b67f-b7119cfe177f admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:55.298361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5d35-3091-43e4-b67f-b7119cfe177f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:55.298837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5d35-3091-43e4-b67f-b7119cfe177f admin admin] PATH_INFO: `/v3/projects/4d9bcbb76be44039a6ebae8686f43b48/users/74d5ece9abcb4b7ea54ca756523080d0/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:55.302577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb5d35-3091-43e4-b67f-b7119cfe177f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9bcbb76be44039a6ebae8686f43b48, user_id=74d5ece9abcb4b7ea54ca756523080d0, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:55.308452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb5d35-3091-43e4-b67f-b7119cfe177f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:55.316032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 843/1676] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:25:54 2022] DELETE /identity/v3/users/85ad54b2480a4ad9b548deb6e2026d5b => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:55.338128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f4e3c-4c66-42f3-8c34-a6cbae501df9 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:55.338787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f4e3c-4c66-42f3-8c34-a6cbae501df9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:55.339306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f4e3c-4c66-42f3-8c34-a6cbae501df9 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:55.342939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 844/1678] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 11 11:25:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:25:55.349142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3be52-2421-4fae-85f4-36075ce8125f None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:55.349772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3be52-2421-4fae-85f4-36075ce8125f None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:55.350443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3be52-2421-4fae-85f4-36075ce8125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:55.358531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 834/1678] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:55 2022] PUT /identity/v3/projects/4d9bcbb76be44039a6ebae8686f43b48/users/74d5ece9abcb4b7ea54ca756523080d0/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:55.367616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f87f2-629d-472a-9647-35e206feffac None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:55.374882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f87f2-629d-472a-9647-35e206feffac None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:55.374882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f87f2-629d-472a-9647-35e206feffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:55.447606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:55.451393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-81f3be52-2421-4fae-85f4-36075ce8125f None None] MFA Rules not processed for user `cb1fa1e11b1a44aaa31cc19f7c199285`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:55.455299 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-305f87f2-629d-472a-9647-35e206feffac None None] MFA Rules not processed for user `74d5ece9abcb4b7ea54ca756523080d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:55.456128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-81f3be52-2421-4fae-85f4-36075ce8125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:55.459167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-305f87f2-629d-472a-9647-35e206feffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:55.687244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 845/1679] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:25:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:25:55.720742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 835/1680] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:55.733319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3be52-2421-4fae-85f4-36075ce8125f None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:55.734819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-81f3be52-2421-4fae-85f4-36075ce8125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:55.802791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:55.826975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f87f2-629d-472a-9647-35e206feffac None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:55.829455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-305f87f2-629d-472a-9647-35e206feffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:55.862923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f87f2-629d-472a-9647-35e206feffac None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:55.888485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:55.891121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:55.895211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:55.895874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:55.948722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:55.954505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:55.981721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 836/1681] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:55.982998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:55.984854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:55.990826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3be52-2421-4fae-85f4-36075ce8125f None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:56.005023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb234e2-1457-4f77-8a67-ec040861635e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:56.012774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb234e2-1457-4f77-8a67-ec040861635e service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:56.013080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb234e2-1457-4f77-8a67-ec040861635e service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:56.013362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb234e2-1457-4f77-8a67-ec040861635e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:56.038635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb234e2-1457-4f77-8a67-ec040861635e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:56.040253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb234e2-1457-4f77-8a67-ec040861635e service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:56.078586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 846/1682] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:56.231243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc2250-ab88-4553-8dcf-19a5b94eeedb service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:56.238739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:56.240203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:56.240488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:56.241584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] PATH_INFO: `/v3/projects/553c41f9111e4be28dbb33b2d6d548bb` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:56.243291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=553c41f9111e4be28dbb33b2d6d548bb)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:56.245431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:56.311628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb234e2-1457-4f77-8a67-ec040861635e service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:56.324658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb234e2-1457-4f77-8a67-ec040861635e service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:56.330560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:56.331847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:56.332211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:56.332573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] PATH_INFO: `/v3/users/8600b1c9f8744f9da418d06e9fcf5d90` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:56.334047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8600b1c9f8744f9da418d06e9fcf5d90)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:56.336117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:56.520392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553c41f9111e4be28dbb33b2d6d548bb', 'request_id': 'req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.538393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553c41f9111e4be28dbb33b2d6d548bb', 'request_id': 'req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.711184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553c41f9111e4be28dbb33b2d6d548bb', 'request_id': 'req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.726798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8600b1c9f8744f9da418d06e9fcf5d90'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.728233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8600b1c9f8744f9da418d06e9fcf5d90'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.729860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8600b1c9f8744f9da418d06e9fcf5d90'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.734761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] The token cache is being invalidate because project 553c41f9111e4be28dbb33b2d6d548bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:25:56.747077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.748074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.754182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 837/1683] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:25:55 2022] DELETE /identity/v3/projects/553c41f9111e4be28dbb33b2d6d548bb => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:56.755068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:56.766350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8600b1c9f8744f9da418d06e9fcf5d90'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.772157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:56.796232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-15bc2179-dc00-4a15-9318-10c582a26f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8600b1c9f8744f9da418d06e9fcf5d90'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:56.919709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d14f26-12b0-483e-9c6e-cb16345f6f6e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:56.935493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:56.937849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:56.938140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:56.938421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] PATH_INFO: `/v3/projects/8e7498feab634b50931bbadd2fa80827` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:56.946628 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e7498feab634b50931bbadd2fa80827)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:56.956960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 847/1684] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:25:56 2022] DELETE /identity/v3/users/8600b1c9f8744f9da418d06e9fcf5d90 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:25:56.962586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7fb31-16ea-47ac-ad40-58e68b229be9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:56.962586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7fb31-16ea-47ac-ad40-58e68b229be9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:56.962586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7fb31-16ea-47ac-ad40-58e68b229be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:56.963014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:56.997426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-aab7fb31-16ea-47ac-ad40-58e68b229be9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:56.998665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-aab7fb31-16ea-47ac-ad40-58e68b229be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:57.189734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 848/1685] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:57.189936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e7498feab634b50931bbadd2fa80827', 'request_id': 'req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.199462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ea575-61a5-41ae-acae-51a11fd42f78 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:57.199821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ea575-61a5-41ae-acae-51a11fd42f78 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:57.200110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ea575-61a5-41ae-acae-51a11fd42f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:57.206656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e7498feab634b50931bbadd2fa80827', 'request_id': 'req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.242377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8e8ea575-61a5-41ae-acae-51a11fd42f78 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:57.245312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8e8ea575-61a5-41ae-acae-51a11fd42f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:57.333183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e7498feab634b50931bbadd2fa80827', 'request_id': 'req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.336841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:57.345036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 849/1686] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:57.349155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] The token cache is being invalidate because project 8e7498feab634b50931bbadd2fa80827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:25:57.350073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c84a-f9e5-4ef5-8725-aaad5f9bc3f9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:57.350473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c84a-f9e5-4ef5-8725-aaad5f9bc3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:57.351988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c84a-f9e5-4ef5-8725-aaad5f9bc3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:57.354899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.356252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.362666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 838/1687] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:25:56 2022] DELETE /identity/v3/projects/8e7498feab634b50931bbadd2fa80827 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:57.366328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:57.369132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:57.389892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462d72a-47f2-4b3d-a764-93ddbd3ec39a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:57.404497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-03b4c84a-f9e5-4ef5-8725-aaad5f9bc3f9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:57.408736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:57.408736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:57.408736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:57.408736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] PATH_INFO: `/v3/users/1df400ae412245b3854350028b055794` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:57.409198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-03b4c84a-f9e5-4ef5-8725-aaad5f9bc3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:57.409290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df400ae412245b3854350028b055794)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:57.415314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:57.429973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 850/1688] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:57.435211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4c84a-f9e5-4ef5-8725-aaad5f9bc3f9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:57.500546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4c84a-f9e5-4ef5-8725-aaad5f9bc3f9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:57.527539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:57.528640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:57.535980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:57.536622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:57.549979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:25:57.549979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:57.549979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:57.580914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 851/1689] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:25:57.584928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:57.588221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:57.643553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:57.715463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df400ae412245b3854350028b055794'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.718622 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df400ae412245b3854350028b055794'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.719405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df400ae412245b3854350028b055794'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.740859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001211b9-6ea3-478d-8023-b11361dd472b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:57.754000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df400ae412245b3854350028b055794'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.756669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:57.760333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:57.760734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:57.766447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:57.766447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:57.792929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df400ae412245b3854350028b055794'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:57.805937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:57.807826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:57.821486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 852/1690] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:57.822758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:57.824701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:57.841055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64678160-3ac9-4902-9434-6ca25707c282 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:57.846467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc4dcb-09b0-4bfb-824d-014bdedfa295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:57.847620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4dcb-09b0-4bfb-824d-014bdedfa295 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:57.848014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4dcb-09b0-4bfb-824d-014bdedfa295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:57.848412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4dcb-09b0-4bfb-824d-014bdedfa295 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:57.849319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc4dcb-09b0-4bfb-824d-014bdedfa295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:57.855103 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc4dcb-09b0-4bfb-824d-014bdedfa295 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:57.898012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 839/1691] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:25:57 2022] DELETE /identity/v3/users/1df400ae412245b3854350028b055794 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:25:57.996215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:57.997275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 853/1692] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:25:57 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:58.097770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2877f2b-64b7-406c-8ba2-57bc1896969d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.097770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2877f2b-64b7-406c-8ba2-57bc1896969d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.097770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2877f2b-64b7-406c-8ba2-57bc1896969d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.146516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a2877f2b-64b7-406c-8ba2-57bc1896969d None None] MFA Rules not processed for user `993a59e8746948ab8a0f11d877314e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:58.157226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a2877f2b-64b7-406c-8ba2-57bc1896969d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:58.160452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153397ea-6822-4c14-b74f-f56be3562675 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:58.243812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:58.245409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.245785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.245979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.246948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:58.253984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:58.286461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 854/1693] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:58.287944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210345-486d-4e28-9c08-9ae5c6746836 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.288347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210345-486d-4e28-9c08-9ae5c6746836 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.288516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6210345-486d-4e28-9c08-9ae5c6746836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.330272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b6210345-486d-4e28-9c08-9ae5c6746836 None None] MFA Rules not processed for user `5f9ae0e6a2534bb0b5d5e9327b01b238`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:58.331743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b6210345-486d-4e28-9c08-9ae5c6746836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:58.409445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 840/1694] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:25:57 2022] POST /identity/v3/users => generated 284 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:25:58.410456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:58.419460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:58.535614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 855/1695] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:58.535614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6210345-486d-4e28-9c08-9ae5c6746836 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:58.552080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6210345-486d-4e28-9c08-9ae5c6746836 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:58.565287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039350b-e520-4a27-9c8e-04e276701a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:58.565749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039350b-e520-4a27-9c8e-04e276701a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.566040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039350b-e520-4a27-9c8e-04e276701a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.566315 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039350b-e520-4a27-9c8e-04e276701a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.567430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039350b-e520-4a27-9c8e-04e276701a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:58.568781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039350b-e520-4a27-9c8e-04e276701a03 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:58.596940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 856/1696] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:58.598414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cb8c5-a5c6-4cb9-890d-75492cf1a7ea None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.598776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cb8c5-a5c6-4cb9-890d-75492cf1a7ea None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.598926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cb8c5-a5c6-4cb9-890d-75492cf1a7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.628289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d11442-e1da-4338-b791-b29e756b506c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:58.640873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4b7cb8c5-a5c6-4cb9-890d-75492cf1a7ea None None] MFA Rules not processed for user `5f9ae0e6a2534bb0b5d5e9327b01b238`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:58.645710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4b7cb8c5-a5c6-4cb9-890d-75492cf1a7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:58.650654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937f2d-e7c4-412b-93b2-737ee33ba75c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:58.651514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937f2d-e7c4-412b-93b2-737ee33ba75c service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.651802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937f2d-e7c4-412b-93b2-737ee33ba75c service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.656745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30937f2d-e7c4-412b-93b2-737ee33ba75c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.672513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 857/1697] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:58.674160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cb8c5-a5c6-4cb9-890d-75492cf1a7ea None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:58.676163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30937f2d-e7c4-412b-93b2-737ee33ba75c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:58.679064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4b7cb8c5-a5c6-4cb9-890d-75492cf1a7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:58.682596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30937f2d-e7c4-412b-93b2-737ee33ba75c service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:58.715486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 841/1698] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:58.716481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937f2d-e7c4-412b-93b2-737ee33ba75c service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:58.759058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30937f2d-e7c4-412b-93b2-737ee33ba75c service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:58.773608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56d943-c6ae-4e1a-a411-03e9cb0a0042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:58.773608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d943-c6ae-4e1a-a411-03e9cb0a0042 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.773608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d943-c6ae-4e1a-a411-03e9cb0a0042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.773608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d943-c6ae-4e1a-a411-03e9cb0a0042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.774023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56d943-c6ae-4e1a-a411-03e9cb0a0042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:58.777891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56d943-c6ae-4e1a-a411-03e9cb0a0042 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:58.797811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 842/1699] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:25:58.799152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56d943-c6ae-4e1a-a411-03e9cb0a0042 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:58.818308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56d943-c6ae-4e1a-a411-03e9cb0a0042 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:58.829420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:58.830391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.830679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.830959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] PATH_INFO: `/v3/projects/b8eec917728548c4a830cafd4adab801/users/1b775285acd8456e86d679c20955c76d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.866951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8eec917728548c4a830cafd4adab801, user_id=1b775285acd8456e86d679c20955c76d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:58.870114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:58.941683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:58.949771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cb8c5-a5c6-4cb9-890d-75492cf1a7ea None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:58.955825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2d136-f589-42de-94e2-5da3e38c31ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:58.956825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2d136-f589-42de-94e2-5da3e38c31ff service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:58.957728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2d136-f589-42de-94e2-5da3e38c31ff service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:58.958089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2d136-f589-42de-94e2-5da3e38c31ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:58.961231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f0c2d136-f589-42de-94e2-5da3e38c31ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:58.981506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 843/1700] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:25:58 2022] PUT /identity/v3/projects/b8eec917728548c4a830cafd4adab801/users/1b775285acd8456e86d679c20955c76d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:25:58.982407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:58.987209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:59.005577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2d136-f589-42de-94e2-5da3e38c31ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:59.008268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2d136-f589-42de-94e2-5da3e38c31ff service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:59.025058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 858/1701] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:59.026795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254dd6-19ad-4e5f-b1e4-af6f7791e664 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:59.027360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254dd6-19ad-4e5f-b1e4-af6f7791e664 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:59.027844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b254dd6-19ad-4e5f-b1e4-af6f7791e664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:59.050334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2b254dd6-19ad-4e5f-b1e4-af6f7791e664 None None] MFA Rules not processed for user `e47c365946cf4264a2d2205ace04379d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:59.052556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2b254dd6-19ad-4e5f-b1e4-af6f7791e664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:59.184291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 859/1702] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:59.185774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdfb82-e8cb-47dc-a580-8bc364c44a9d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:59.186078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdfb82-e8cb-47dc-a580-8bc364c44a9d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:59.186415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdfb82-e8cb-47dc-a580-8bc364c44a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:59.214589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a2abc-d143-4454-918e-0d0102536089 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:59.235755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:59.250798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:59.254248 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1ffdfb82-e8cb-47dc-a580-8bc364c44a9d None None] MFA Rules not processed for user `1b775285acd8456e86d679c20955c76d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:59.254584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:59.254786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ffdfb82-e8cb-47dc-a580-8bc364c44a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:59.255397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:59.355807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:25:59.470857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 860/1703] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:25:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:25:59.479432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdfb82-e8cb-47dc-a580-8bc364c44a9d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:59.483900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ffdfb82-e8cb-47dc-a580-8bc364c44a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:59.487467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:59.489038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:59.512604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 844/1704] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:59.513444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:25:59.514961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:59.604757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde3f6c-0f21-4c9e-a6a1-d48d7d8c8e92 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:59.608720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6731a-1c8b-4c9c-8394-d23c036843b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:59.609585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6731a-1c8b-4c9c-8394-d23c036843b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:59.609876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6731a-1c8b-4c9c-8394-d23c036843b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:59.614545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6731a-1c8b-4c9c-8394-d23c036843b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:59.634545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6731a-1c8b-4c9c-8394-d23c036843b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:59.637038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6731a-1c8b-4c9c-8394-d23c036843b2 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:59.681574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 845/1705] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:25:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:25:59.725044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdfb82-e8cb-47dc-a580-8bc364c44a9d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:25:59.740101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:25:59.740101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:59.740101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:59.740524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] PATH_INFO: `/v3/projects/b438e3f541c94c67a5ef6a7d43aa3fb2` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:59.742105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b438e3f541c94c67a5ef6a7d43aa3fb2)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:25:59.747772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:25:59.880367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9192ba1-9dd8-4dfe-bce8-ea0509c73245 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:59.880665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9192ba1-9dd8-4dfe-bce8-ea0509c73245 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:59.880948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9192ba1-9dd8-4dfe-bce8-ea0509c73245 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:59.883838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 846/1706] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 11 11:25:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:25:59.889730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb754b7-a64e-4326-9e81-cee03ae586be None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:25:59.890023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb754b7-a64e-4326-9e81-cee03ae586be None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:25:59.890309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb754b7-a64e-4326-9e81-cee03ae586be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:25:59.948095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cfb754b7-a64e-4326-9e81-cee03ae586be None None] MFA Rules not processed for user `cb1fa1e11b1a44aaa31cc19f7c199285`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:25:59.949784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cfb754b7-a64e-4326-9e81-cee03ae586be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:25:59.980099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b438e3f541c94c67a5ef6a7d43aa3fb2', 'request_id': 'req-f2be9fe5-9809-4c08-b75c-81a58a013a55'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:25:59.994451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b438e3f541c94c67a5ef6a7d43aa3fb2', 'request_id': 'req-f2be9fe5-9809-4c08-b75c-81a58a013a55'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.101926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 847/1707] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:25:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:00.109271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb754b7-a64e-4326-9e81-cee03ae586be None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:00.112235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cfb754b7-a64e-4326-9e81-cee03ae586be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:00.121475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b438e3f541c94c67a5ef6a7d43aa3fb2', 'request_id': 'req-f2be9fe5-9809-4c08-b75c-81a58a013a55'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.143314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] The token cache is being invalidate because project b438e3f541c94c67a5ef6a7d43aa3fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:00.143616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.147935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.154916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 861/1708] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:25:59 2022] DELETE /identity/v3/projects/b438e3f541c94c67a5ef6a7d43aa3fb2 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:00.159318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:00.164095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:00.317224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb754b7-a64e-4326-9e81-cee03ae586be None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:00.335040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:00.336175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:00.336639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:00.337129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] PATH_INFO: `/v3/projects/cfba21ecf40a42d08cb398fad71783f6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:00.341271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfba21ecf40a42d08cb398fad71783f6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:00.346437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:00.354515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be9fe5-9809-4c08-b75c-81a58a013a55 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:00.366110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:00.367153 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:00.367464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:00.369092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] PATH_INFO: `/v3/users/74d5ece9abcb4b7ea54ca756523080d0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:00.372324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d5ece9abcb4b7ea54ca756523080d0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:00.376308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:00.588476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfba21ecf40a42d08cb398fad71783f6', 'request_id': 'req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.605791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfba21ecf40a42d08cb398fad71783f6', 'request_id': 'req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.727201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfba21ecf40a42d08cb398fad71783f6', 'request_id': 'req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.745106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] The token cache is being invalidate because project cfba21ecf40a42d08cb398fad71783f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:00.745415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.747573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.755681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 848/1709] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:00 2022] DELETE /identity/v3/projects/cfba21ecf40a42d08cb398fad71783f6 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:00.755681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:00.756777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d5ece9abcb4b7ea54ca756523080d0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.758582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:00.759621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d5ece9abcb4b7ea54ca756523080d0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.761631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d5ece9abcb4b7ea54ca756523080d0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.782430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d5ece9abcb4b7ea54ca756523080d0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.819940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1086cfcc-0cd4-4680-9c72-7cbf0187592a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d5ece9abcb4b7ea54ca756523080d0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:00.888730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e901a-acd2-44ef-af4b-46ab86dfdda5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:00.933831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 862/1710] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:00 2022] DELETE /identity/v3/users/74d5ece9abcb4b7ea54ca756523080d0 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:00.939540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70a282-359b-4772-aa1b-407c3ebcf47c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:00.939841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70a282-359b-4772-aa1b-407c3ebcf47c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:00.940101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70a282-359b-4772-aa1b-407c3ebcf47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:00.942950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc274226-6068-4eb2-b75e-bc18e46575a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:00.945184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274226-6068-4eb2-b75e-bc18e46575a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:00.945184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274226-6068-4eb2-b75e-bc18e46575a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:00.945184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc274226-6068-4eb2-b75e-bc18e46575a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:00.973347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cc274226-6068-4eb2-b75e-bc18e46575a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:01.023075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274226-6068-4eb2-b75e-bc18e46575a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:01.024381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc274226-6068-4eb2-b75e-bc18e46575a5 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:01.045951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3f70a282-359b-4772-aa1b-407c3ebcf47c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:01.047211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3f70a282-359b-4772-aa1b-407c3ebcf47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:01.065631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 849/1711] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:01.101144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:01.285793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 863/1712] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:01.293418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc44512-080a-4d1e-835e-4c8237bbd0f7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:01.294063 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc44512-080a-4d1e-835e-4c8237bbd0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:01.294393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc44512-080a-4d1e-835e-4c8237bbd0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:01.321948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4fc44512-080a-4d1e-835e-4c8237bbd0f7 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:01.323994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4fc44512-080a-4d1e-835e-4c8237bbd0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:01.340695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70a282-359b-4772-aa1b-407c3ebcf47c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:01.346133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 850/1713] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:01.353021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70a282-359b-4772-aa1b-407c3ebcf47c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:01.382286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:01.382658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:01.382871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:01.383158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:01.385805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:01.386129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:01.395000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:01.411174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 864/1714] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:26:01.415277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc44512-080a-4d1e-835e-4c8237bbd0f7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:01.430516 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc44512-080a-4d1e-835e-4c8237bbd0f7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:01.440366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:01.447752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b514579-4732-40b8-9bf8-b63d8a913afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:01.448567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514579-4732-40b8-9bf8-b63d8a913afc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:01.449098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514579-4732-40b8-9bf8-b63d8a913afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:01.449713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b514579-4732-40b8-9bf8-b63d8a913afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:01.451869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b514579-4732-40b8-9bf8-b63d8a913afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:01.455443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b514579-4732-40b8-9bf8-b63d8a913afc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:01.649315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 851/1715] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:01 2022] POST /identity/v3/projects => generated 369 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:01.650917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:01.686806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b23b3a-22b4-49d2-bcfe-8da83fd2ade3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:01.896172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdecc-0722-4b05-b2ab-6726f07b0e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:01.896923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdecc-0722-4b05-b2ab-6726f07b0e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:01.897217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdecc-0722-4b05-b2ab-6726f07b0e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:01.903018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdecc-0722-4b05-b2ab-6726f07b0e9a admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:01.911006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecdecc-0722-4b05-b2ab-6726f07b0e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:01.923175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecdecc-0722-4b05-b2ab-6726f07b0e9a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:01.983873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e6ce-1834-4549-b6d6-9e367e109578 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:01.984196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e6ce-1834-4549-b6d6-9e367e109578 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:01.984811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01e6ce-1834-4549-b6d6-9e367e109578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:02.079146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fb01e6ce-1834-4549-b6d6-9e367e109578 None None] MFA Rules not processed for user `cbcc15c786fb43669ac34a4897735c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:02.081214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fb01e6ce-1834-4549-b6d6-9e367e109578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:02.095072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 865/1716] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:01 2022] POST /identity/v3/users => generated 285 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:02.099879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdecc-0722-4b05-b2ab-6726f07b0e9a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:02.149175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdecc-0722-4b05-b2ab-6726f07b0e9a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:02.167347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:02.170330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:02.170628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:02.170972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:02.171767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:02.184228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:02.240391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 852/1717] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:02.241305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01e6ce-1834-4549-b6d6-9e367e109578 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:02.250028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 866/1718] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:02.272563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:02.279513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:02.292460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01e6ce-1834-4549-b6d6-9e367e109578 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:02.311981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440369-9020-4a7e-9a81-8b3e1da37de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:02.312936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440369-9020-4a7e-9a81-8b3e1da37de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:02.313382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440369-9020-4a7e-9a81-8b3e1da37de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:02.313819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440369-9020-4a7e-9a81-8b3e1da37de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:02.315942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da440369-9020-4a7e-9a81-8b3e1da37de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:02.321278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da440369-9020-4a7e-9a81-8b3e1da37de3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:02.371759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 853/1719] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:02.376048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440369-9020-4a7e-9a81-8b3e1da37de3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:02.405830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440369-9020-4a7e-9a81-8b3e1da37de3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:02.430762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9167600-683b-46ab-befd-4cc629fd0567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:02.431844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9167600-683b-46ab-befd-4cc629fd0567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:02.432130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9167600-683b-46ab-befd-4cc629fd0567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:02.432401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9167600-683b-46ab-befd-4cc629fd0567 admin admin] PATH_INFO: `/v3/projects/6d58d32268d64d629caf5f4e7dfca47e/users/27ee99de8efd41cb9fa7268e10bbadb3/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:02.486909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9167600-683b-46ab-befd-4cc629fd0567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d58d32268d64d629caf5f4e7dfca47e, user_id=27ee99de8efd41cb9fa7268e10bbadb3, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:02.501114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9167600-683b-46ab-befd-4cc629fd0567 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:02.580645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86b7d9-ec2c-4fd5-a149-3387ef851500 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:02.606433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:02.608004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:02.608858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:02.609336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:02.622770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:02.634474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:02.643869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 854/1720] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:02 2022] PUT /identity/v3/projects/6d58d32268d64d629caf5f4e7dfca47e/users/27ee99de8efd41cb9fa7268e10bbadb3/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:02.655908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 867/1721] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:26:02.659445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d98f4b-a837-412d-8db3-02b01dcc75fc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:02.659827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d98f4b-a837-412d-8db3-02b01dcc75fc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:02.660183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d98f4b-a837-412d-8db3-02b01dcc75fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:02.719906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-44d98f4b-a837-412d-8db3-02b01dcc75fc None None] MFA Rules not processed for user `27ee99de8efd41cb9fa7268e10bbadb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:02.721697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-44d98f4b-a837-412d-8db3-02b01dcc75fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:02.773645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:02.928325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:02.930215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:02.956741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0519-65c3-4563-892a-fd62f6e10cc1 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:02.958043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 855/1722] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:02.962950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d98f4b-a837-412d-8db3-02b01dcc75fc None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:03.080443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:03.100290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d98f4b-a837-412d-8db3-02b01dcc75fc None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:03.107820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9ac5c-915c-41fa-8739-142eef5f277a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:03.108743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9ac5c-915c-41fa-8739-142eef5f277a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:03.109488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9ac5c-915c-41fa-8739-142eef5f277a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:03.109932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9ac5c-915c-41fa-8739-142eef5f277a admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:03.111704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9ac5c-915c-41fa-8739-142eef5f277a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:03.114680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9ac5c-915c-41fa-8739-142eef5f277a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:03.283590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 856/1723] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:02 2022] POST /identity/v3/projects => generated 369 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:03.287772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9ac5c-915c-41fa-8739-142eef5f277a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:03.292127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:03.294074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:03.311024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:03.311562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] PATH_INFO: `/v3/projects/4d9bcbb76be44039a6ebae8686f43b48` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:03.317133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d9bcbb76be44039a6ebae8686f43b48)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:03.323754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:03.523616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9ac5c-915c-41fa-8739-142eef5f277a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:03.535238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf57b7-f1c6-4dfd-b40e-7520e3e67e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:03.536208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf57b7-f1c6-4dfd-b40e-7520e3e67e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:03.536645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf57b7-f1c6-4dfd-b40e-7520e3e67e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:03.537347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf57b7-f1c6-4dfd-b40e-7520e3e67e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:03.538226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf57b7-f1c6-4dfd-b40e-7520e3e67e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:03.540368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf57b7-f1c6-4dfd-b40e-7520e3e67e2d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:03.649384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d9bcbb76be44039a6ebae8686f43b48', 'request_id': 'req-4487b47b-f8ba-42eb-888a-9e36f039cfe6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:03.673066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d9bcbb76be44039a6ebae8686f43b48', 'request_id': 'req-4487b47b-f8ba-42eb-888a-9e36f039cfe6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:03.756601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 857/1724] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:03 2022] POST /identity/v3/users => generated 291 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:03.758229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010334dd-a369-42f2-9ac9-1c698a85e6f5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:03.758747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010334dd-a369-42f2-9ac9-1c698a85e6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:03.759346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010334dd-a369-42f2-9ac9-1c698a85e6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:03.802404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-010334dd-a369-42f2-9ac9-1c698a85e6f5 None None] MFA Rules not processed for user `15ff2cacb5c1450482597a2f5fa3c8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:03.803832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-010334dd-a369-42f2-9ac9-1c698a85e6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:03.842416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d9bcbb76be44039a6ebae8686f43b48', 'request_id': 'req-4487b47b-f8ba-42eb-888a-9e36f039cfe6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:03.880617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] The token cache is being invalidate because project 4d9bcbb76be44039a6ebae8686f43b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:03.880903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:03.889064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:03.894664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 868/1725] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:02 2022] DELETE /identity/v3/projects/4d9bcbb76be44039a6ebae8686f43b48 => generated 0 bytes in 963 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:03.896095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:03.897762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:03.949012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 858/1726] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:03.951017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f78b-ab80-49df-8ef3-cfda04b2f515 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:03.951609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f78b-ab80-49df-8ef3-cfda04b2f515 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:03.952097 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7f78b-ab80-49df-8ef3-cfda04b2f515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:03.986753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d4a7f78b-ab80-49df-8ef3-cfda04b2f515 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:03.988341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d4a7f78b-ab80-49df-8ef3-cfda04b2f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:04.017642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:04.107868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487b47b-f8ba-42eb-888a-9e36f039cfe6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:04.122826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600612d2-1019-4abf-9b04-199b1b6d27fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:04.127785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600612d2-1019-4abf-9b04-199b1b6d27fa admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.130904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600612d2-1019-4abf-9b04-199b1b6d27fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.131221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600612d2-1019-4abf-9b04-199b1b6d27fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.138276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600612d2-1019-4abf-9b04-199b1b6d27fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:04.139399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600612d2-1019-4abf-9b04-199b1b6d27fa admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:04.149708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 859/1727] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:04.152286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7f78b-ab80-49df-8ef3-cfda04b2f515 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:04.154755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d4a7f78b-ab80-49df-8ef3-cfda04b2f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:04.174089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 869/1728] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:04.175783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23335c5a-6f47-4304-933a-aeb9c883a537 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.176283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23335c5a-6f47-4304-933a-aeb9c883a537 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.176979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23335c5a-6f47-4304-933a-aeb9c883a537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.202252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-23335c5a-6f47-4304-933a-aeb9c883a537 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:04.204149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-23335c5a-6f47-4304-933a-aeb9c883a537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:04.216175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 870/1729] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:04.221316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23335c5a-6f47-4304-933a-aeb9c883a537 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:04.231531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23335c5a-6f47-4304-933a-aeb9c883a537 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:04.238293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3214b-e81c-4414-821e-84aeb2f49513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:04.239553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3214b-e81c-4414-821e-84aeb2f49513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.240040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3214b-e81c-4414-821e-84aeb2f49513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.240286 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:04.240860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3214b-e81c-4414-821e-84aeb2f49513 admin admin] PATH_INFO: `/v3/projects/e360c94bd98b442aae832d2542c9cc60/users/f05899d20a284ec2adb223dc10331a97/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.261707 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3214b-e81c-4414-821e-84aeb2f49513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e360c94bd98b442aae832d2542c9cc60, user_id=f05899d20a284ec2adb223dc10331a97, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:04.265352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3214b-e81c-4414-821e-84aeb2f49513 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:04.345727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7f78b-ab80-49df-8ef3-cfda04b2f515 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:04.363050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d478167-2ca3-4f81-868f-103fb5a22e31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:04.363827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d478167-2ca3-4f81-868f-103fb5a22e31 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.364097 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d478167-2ca3-4f81-868f-103fb5a22e31 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.364386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d478167-2ca3-4f81-868f-103fb5a22e31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.386290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 871/1730] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:04 2022] PUT /identity/v3/projects/e360c94bd98b442aae832d2542c9cc60/users/f05899d20a284ec2adb223dc10331a97/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:04.390272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3214b-e81c-4414-821e-84aeb2f49513 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:04.398863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d478167-2ca3-4f81-868f-103fb5a22e31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:04.408201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d478167-2ca3-4f81-868f-103fb5a22e31 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:04.503175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 860/1731] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:04.506034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d478167-2ca3-4f81-868f-103fb5a22e31 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:04.568408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d478167-2ca3-4f81-868f-103fb5a22e31 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:04.583032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3214b-e81c-4414-821e-84aeb2f49513 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:04.587340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de027a1-dbbc-4b5a-824c-178e6fbe3509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:04.591730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de027a1-dbbc-4b5a-824c-178e6fbe3509 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.591730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de027a1-dbbc-4b5a-824c-178e6fbe3509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.591730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de027a1-dbbc-4b5a-824c-178e6fbe3509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.591730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de027a1-dbbc-4b5a-824c-178e6fbe3509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:04.593603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de027a1-dbbc-4b5a-824c-178e6fbe3509 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:04.595857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:04.597316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.597773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.599992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.601227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:04.601810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:04.617598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:04.637760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 861/1732] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:04.648135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de027a1-dbbc-4b5a-824c-178e6fbe3509 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:04.669517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 872/1733] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:04.677817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:04.694021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de027a1-dbbc-4b5a-824c-178e6fbe3509 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:04.711035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5dfc-92a1-4cde-97f5-80f1a8ef5311 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:04.723330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74040e41-7449-43ac-8de9-ccd2f36ef1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:04.724259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040e41-7449-43ac-8de9-ccd2f36ef1fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.724613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040e41-7449-43ac-8de9-ccd2f36ef1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.725038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74040e41-7449-43ac-8de9-ccd2f36ef1fe admin admin] PATH_INFO: `/v3/projects/e360c94bd98b442aae832d2542c9cc60/users/f05899d20a284ec2adb223dc10331a97/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.725725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6e8fa-cd52-4a45-ad85-d689aa1230b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:04.726629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6e8fa-cd52-4a45-ad85-d689aa1230b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.727105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6e8fa-cd52-4a45-ad85-d689aa1230b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.727582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74040e41-7449-43ac-8de9-ccd2f36ef1fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e360c94bd98b442aae832d2542c9cc60, user_id=f05899d20a284ec2adb223dc10331a97, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:04.727989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6e8fa-cd52-4a45-ad85-d689aa1230b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.728889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6e8fa-cd52-4a45-ad85-d689aa1230b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:04.730517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74040e41-7449-43ac-8de9-ccd2f36ef1fe admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:04.733915 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6e8fa-cd52-4a45-ad85-d689aa1230b7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:04.778117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 862/1734] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:04 2022] PUT /identity/v3/projects/e360c94bd98b442aae832d2542c9cc60/users/f05899d20a284ec2adb223dc10331a97/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:04.797086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa42b0-6e94-45c5-9f22-89b4ab36fc40 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:04.797519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa42b0-6e94-45c5-9f22-89b4ab36fc40 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:04.797873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa42b0-6e94-45c5-9f22-89b4ab36fc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:04.882811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 873/1735] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:04 2022] POST /identity/v3/projects => generated 365 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:04.884121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-80fa42b0-6e94-45c5-9f22-89b4ab36fc40 None None] MFA Rules not processed for user `f05899d20a284ec2adb223dc10331a97`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:04.884798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6e8fa-cd52-4a45-ad85-d689aa1230b7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:04.885652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-80fa42b0-6e94-45c5-9f22-89b4ab36fc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:05.059360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6e8fa-cd52-4a45-ad85-d689aa1230b7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:05.085522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce7acc-268d-4593-95c5-5fa25e8167f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:05.087502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce7acc-268d-4593-95c5-5fa25e8167f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:05.091206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce7acc-268d-4593-95c5-5fa25e8167f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:05.091870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce7acc-268d-4593-95c5-5fa25e8167f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:05.096213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce7acc-268d-4593-95c5-5fa25e8167f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:05.101280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 863/1736] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:04 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:05.102298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce7acc-268d-4593-95c5-5fa25e8167f2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:05.118697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa42b0-6e94-45c5-9f22-89b4ab36fc40 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:05.120166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-80fa42b0-6e94-45c5-9f22-89b4ab36fc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:05.270162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 874/1737] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:04 2022] POST /identity/v3/users => generated 283 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:05.271211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce7acc-268d-4593-95c5-5fa25e8167f2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:05.301353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce7acc-268d-4593-95c5-5fa25e8167f2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:05.306859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332f098-df8d-4ba3-9ce2-dcf58fab3762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:05.307586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332f098-df8d-4ba3-9ce2-dcf58fab3762 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:05.307873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332f098-df8d-4ba3-9ce2-dcf58fab3762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:05.308148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332f098-df8d-4ba3-9ce2-dcf58fab3762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:05.308944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332f098-df8d-4ba3-9ce2-dcf58fab3762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:05.310818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332f098-df8d-4ba3-9ce2-dcf58fab3762 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:05.357158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa42b0-6e94-45c5-9f22-89b4ab36fc40 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:05.359746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 875/1738] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:05.363376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332f098-df8d-4ba3-9ce2-dcf58fab3762 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:05.387850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a5c71-ed7d-4ea0-a820-6bfef23737b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:05.389118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a5c71-ed7d-4ea0-a820-6bfef23737b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:05.389554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a5c71-ed7d-4ea0-a820-6bfef23737b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:05.389979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a5c71-ed7d-4ea0-a820-6bfef23737b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:05.437806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a5c71-ed7d-4ea0-a820-6bfef23737b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:05.439630 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a5c71-ed7d-4ea0-a820-6bfef23737b4 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:05.441553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332f098-df8d-4ba3-9ce2-dcf58fab3762 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:05.456423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bb8f3-eeed-49e1-a534-92ae45274964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:05.458354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bb8f3-eeed-49e1-a534-92ae45274964 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:05.460154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bb8f3-eeed-49e1-a534-92ae45274964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:05.460154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bb8f3-eeed-49e1-a534-92ae45274964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:05.460935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bb8f3-eeed-49e1-a534-92ae45274964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:05.469888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223bb8f3-eeed-49e1-a534-92ae45274964 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:05.502880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 864/1739] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:05 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:26:05.510434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a5c71-ed7d-4ea0-a820-6bfef23737b4 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:05.512864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 876/1740] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:05.575233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a5c71-ed7d-4ea0-a820-6bfef23737b4 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:05.601521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b050bc-2253-45d2-863a-149f01bab0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:05.602434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b050bc-2253-45d2-863a-149f01bab0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:05.602719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b050bc-2253-45d2-863a-149f01bab0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:05.602998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b050bc-2253-45d2-863a-149f01bab0c7 admin admin] PATH_INFO: `/v3/projects/16cb671b750e4052aa2b2bdb3d8a26d9/users/9d1535cb30a547358ec7197185ffd67d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:05.687221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b050bc-2253-45d2-863a-149f01bab0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cb671b750e4052aa2b2bdb3d8a26d9, user_id=9d1535cb30a547358ec7197185ffd67d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:05.698173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b050bc-2253-45d2-863a-149f01bab0c7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:05.711057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db92368-861f-4f74-8a6b-300d09ae57d2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:05.711608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db92368-861f-4f74-8a6b-300d09ae57d2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:05.712089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db92368-861f-4f74-8a6b-300d09ae57d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:05.871304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 865/1741] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:05 2022] PUT /identity/v3/projects/16cb671b750e4052aa2b2bdb3d8a26d9/users/9d1535cb30a547358ec7197185ffd67d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:05.872249 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b050bc-2253-45d2-863a-149f01bab0c7 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:05.948167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3db92368-861f-4f74-8a6b-300d09ae57d2 None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:05.958484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3db92368-861f-4f74-8a6b-300d09ae57d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:06.082751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b050bc-2253-45d2-863a-149f01bab0c7 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:06.095762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:06.096496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:06.096800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:06.097061 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:06.101755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:06.178461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 877/1742] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:26:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:06.195300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeba70e-c1ce-4c88-abde-d6218bb8f607 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:06.195612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeba70e-c1ce-4c88-abde-d6218bb8f607 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:06.202483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeba70e-c1ce-4c88-abde-d6218bb8f607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:06.295520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-beeba70e-c1ce-4c88-abde-d6218bb8f607 None None] MFA Rules not processed for user `9d1535cb30a547358ec7197185ffd67d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:06.307280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-beeba70e-c1ce-4c88-abde-d6218bb8f607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:06.328130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:06.332457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:06.367372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 866/1743] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:06.538335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 878/1744] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:06.542585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:06.701352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de641d-e9a2-47b7-8a5b-ada16eb08097 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:06.708454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:06.709327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:06.709749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:06.710158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:06.711715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:06.713791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:06.857259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0a6d-90c4-41de-9085-ae6f68711657 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:06.857259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0a6d-90c4-41de-9085-ae6f68711657 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:06.857259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0a6d-90c4-41de-9085-ae6f68711657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:06.878114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f0ac0a6d-90c4-41de-9085-ae6f68711657 None None] MFA Rules not processed for user `993a59e8746948ab8a0f11d877314e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:06.879689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f0ac0a6d-90c4-41de-9085-ae6f68711657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:06.948241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 867/1745] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:06 2022] POST /identity/v3/projects => generated 365 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:06.949625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:06.957893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:07.003109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 879/1746] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:07.004165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0a6d-90c4-41de-9085-ae6f68711657 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:07.043860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0a6d-90c4-41de-9085-ae6f68711657 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:07.160270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca0ffc-d8a6-47dc-903e-b42d5315f595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:07.161081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca0ffc-d8a6-47dc-903e-b42d5315f595 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:07.161508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca0ffc-d8a6-47dc-903e-b42d5315f595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:07.161905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca0ffc-d8a6-47dc-903e-b42d5315f595 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:07.164134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca0ffc-d8a6-47dc-903e-b42d5315f595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:07.166577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca0ffc-d8a6-47dc-903e-b42d5315f595 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:07.236595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c004b54-4e97-4819-abbf-3aa31d15ebcc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:07.243775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094f654-928b-4061-81e6-955d8d1a2ada service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:07.244452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f654-928b-4061-81e6-955d8d1a2ada service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:07.244741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f654-928b-4061-81e6-955d8d1a2ada service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:07.245025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f654-928b-4061-81e6-955d8d1a2ada service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:07.282746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:07.291276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 880/1747] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:07 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:07.296487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca0ffc-d8a6-47dc-903e-b42d5315f595 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:07.327137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca0ffc-d8a6-47dc-903e-b42d5315f595 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:07.339981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e835f4-5615-4429-a615-3526d688d74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:07.340564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e835f4-5615-4429-a615-3526d688d74e admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:07.340852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e835f4-5615-4429-a615-3526d688d74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:07.341132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e835f4-5615-4429-a615-3526d688d74e admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:07.343203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e835f4-5615-4429-a615-3526d688d74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:07.348585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e835f4-5615-4429-a615-3526d688d74e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:07.397941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 881/1748] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:26:07.399590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e835f4-5615-4429-a615-3526d688d74e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:07.420891 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e835f4-5615-4429-a615-3526d688d74e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:07.427697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019a923-fad5-43ba-9608-a075923b9695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:07.428880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019a923-fad5-43ba-9608-a075923b9695 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:07.429430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019a923-fad5-43ba-9608-a075923b9695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:07.429926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019a923-fad5-43ba-9608-a075923b9695 admin admin] PATH_INFO: `/v3/projects/d7f3c17f45ef434492e19fad56219d7f/users/de02739fb4d94df0a48293b7c22586b6/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:07.431913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094f654-928b-4061-81e6-955d8d1a2ada service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:07.433633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094f654-928b-4061-81e6-955d8d1a2ada service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:07.455739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 868/1749] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 11:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:26:07.478549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019a923-fad5-43ba-9608-a075923b9695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f3c17f45ef434492e19fad56219d7f, user_id=de02739fb4d94df0a48293b7c22586b6, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:07.484022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019a923-fad5-43ba-9608-a075923b9695 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:07.492895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:07.610621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 882/1750] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:07 2022] PUT /identity/v3/projects/d7f3c17f45ef434492e19fad56219d7f/users/de02739fb4d94df0a48293b7c22586b6/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:07.618962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094f654-928b-4061-81e6-955d8d1a2ada service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:07.635391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094f654-928b-4061-81e6-955d8d1a2ada service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:07.773201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194cb7f-e55e-4440-b5f7-bb8642c6f8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:07.773906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194cb7f-e55e-4440-b5f7-bb8642c6f8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:07.774903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194cb7f-e55e-4440-b5f7-bb8642c6f8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:07.774903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194cb7f-e55e-4440-b5f7-bb8642c6f8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:07.785602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194cb7f-e55e-4440-b5f7-bb8642c6f8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:07.792069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194cb7f-e55e-4440-b5f7-bb8642c6f8d2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:07.834754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 869/1751] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:07.836061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019a923-fad5-43ba-9608-a075923b9695 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:07.869418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019a923-fad5-43ba-9608-a075923b9695 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:07.883899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09055d6f-370b-42ff-a29e-b31ce2946066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:07.883899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09055d6f-370b-42ff-a29e-b31ce2946066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:07.883899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09055d6f-370b-42ff-a29e-b31ce2946066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:07.883899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09055d6f-370b-42ff-a29e-b31ce2946066 admin admin] PATH_INFO: `/v3/projects/d7f3c17f45ef434492e19fad56219d7f/users/de02739fb4d94df0a48293b7c22586b6/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:07.893450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09055d6f-370b-42ff-a29e-b31ce2946066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f3c17f45ef434492e19fad56219d7f, user_id=de02739fb4d94df0a48293b7c22586b6, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:07.900833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09055d6f-370b-42ff-a29e-b31ce2946066 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:07.948012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 883/1752] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:07 2022] PUT /identity/v3/projects/d7f3c17f45ef434492e19fad56219d7f/users/de02739fb4d94df0a48293b7c22586b6/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:07.958395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd140-d71a-4070-8f21-81b3a45035e9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:07.958889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd140-d71a-4070-8f21-81b3a45035e9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:07.959536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fd140-d71a-4070-8f21-81b3a45035e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:08.025193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-be3fd140-d71a-4070-8f21-81b3a45035e9 None None] MFA Rules not processed for user `de02739fb4d94df0a48293b7c22586b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:08.027284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-be3fd140-d71a-4070-8f21-81b3a45035e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:08.196922 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 870/1753] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:08.217684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09055d6f-370b-42ff-a29e-b31ce2946066 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:08.436601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09055d6f-370b-42ff-a29e-b31ce2946066 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:08.444651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9febbb1-f2f7-4b38-bbba-d27824973f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:08.447601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febbb1-f2f7-4b38-bbba-d27824973f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:08.449724 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febbb1-f2f7-4b38-bbba-d27824973f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:08.451176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9febbb1-f2f7-4b38-bbba-d27824973f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:08.485096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9febbb1-f2f7-4b38-bbba-d27824973f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:08.489810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9febbb1-f2f7-4b38-bbba-d27824973f6e service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:08.575293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 884/1754] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:09.384860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fd140-d71a-4070-8f21-81b3a45035e9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:09.399693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fd140-d71a-4070-8f21-81b3a45035e9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:09.413121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17404d5-6684-4a6a-9de8-61dc9bab14a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:09.418054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfd545-2e55-474f-9e81-2f7ca12eaa77 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:09.418054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfd545-2e55-474f-9e81-2f7ca12eaa77 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:09.418054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfd545-2e55-474f-9e81-2f7ca12eaa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:09.438603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17404d5-6684-4a6a-9de8-61dc9bab14a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:09.442976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17404d5-6684-4a6a-9de8-61dc9bab14a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:09.449491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17404d5-6684-4a6a-9de8-61dc9bab14a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:09.484204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:09.505466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5bbfd545-2e55-474f-9e81-2f7ca12eaa77 None None] MFA Rules not processed for user `cb1fa1e11b1a44aaa31cc19f7c199285`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:09.514263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5bbfd545-2e55-474f-9e81-2f7ca12eaa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:09.699323 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 885/1755] 10.222.0.46 () {58 vars in 1015 bytes} [Mon Jul 11 11:26:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:09.705819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17404d5-6684-4a6a-9de8-61dc9bab14a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:09.713398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17404d5-6684-4a6a-9de8-61dc9bab14a5 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:09.753855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 871/1756] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:09.959303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfd545-2e55-474f-9e81-2f7ca12eaa77 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:09.961765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5bbfd545-2e55-474f-9e81-2f7ca12eaa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:09.988431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfd545-2e55-474f-9e81-2f7ca12eaa77 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:10.008112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82910676-b00b-4230-b8f9-f96e25d789e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:10.013362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82910676-b00b-4230-b8f9-f96e25d789e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:10.014290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82910676-b00b-4230-b8f9-f96e25d789e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:10.017044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82910676-b00b-4230-b8f9-f96e25d789e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:10.043001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82910676-b00b-4230-b8f9-f96e25d789e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:10.060102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82910676-b00b-4230-b8f9-f96e25d789e9 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:10.106309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 886/1757] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:10.253526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:11.285272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:12.065092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:12.804941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:14.135104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977553-9803-4ab5-9107-0a60af1709b8 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:14.135907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977553-9803-4ab5-9107-0a60af1709b8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:14.135907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977553-9803-4ab5-9107-0a60af1709b8 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:14.137678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 872/1758] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:26:14.191363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb921-1573-41fb-a816-a39c4b76eb7c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:14.192158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb921-1573-41fb-a816-a39c4b76eb7c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:14.192158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb921-1573-41fb-a816-a39c4b76eb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:14.278502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6fddb921-1573-41fb-a816-a39c4b76eb7c None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:14.288395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6fddb921-1573-41fb-a816-a39c4b76eb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:14.537454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 887/1759] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:26:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:14.637081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67977553-9803-4ab5-9107-0a60af1709b8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:14.662704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67977553-9803-4ab5-9107-0a60af1709b8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:14.685007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c0d62-fc9e-4906-b3c1-13bd8e70b2de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:14.692638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0d62-fc9e-4906-b3c1-13bd8e70b2de service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:14.693436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0d62-fc9e-4906-b3c1-13bd8e70b2de service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:14.694123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0d62-fc9e-4906-b3c1-13bd8e70b2de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:14.719516 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c0d62-fc9e-4906-b3c1-13bd8e70b2de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:14.731801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c0d62-fc9e-4906-b3c1-13bd8e70b2de service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:14.767862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 873/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:14.859059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:14.978665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:15.487287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddb921-1573-41fb-a816-a39c4b76eb7c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:15.631287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddb921-1573-41fb-a816-a39c4b76eb7c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:15.648508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:15.649776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:15.650320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:15.650774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] PATH_INFO: `/v3/users/27ee99de8efd41cb9fa7268e10bbadb3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:15.661843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ee99de8efd41cb9fa7268e10bbadb3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:15.664587 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:15.887900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ee99de8efd41cb9fa7268e10bbadb3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:15.890281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ee99de8efd41cb9fa7268e10bbadb3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:15.891284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ee99de8efd41cb9fa7268e10bbadb3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:15.906972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ee99de8efd41cb9fa7268e10bbadb3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:15.928568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ee99de8efd41cb9fa7268e10bbadb3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:16.034304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 888/1761] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:15 2022] DELETE /identity/v3/users/27ee99de8efd41cb9fa7268e10bbadb3 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:16.035650 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56181963-0c93-4884-b8e5-874050a59c45 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:16.036176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56181963-0c93-4884-b8e5-874050a59c45 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:16.036254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56181963-0c93-4884-b8e5-874050a59c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:16.085086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-56181963-0c93-4884-b8e5-874050a59c45 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:16.087081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-56181963-0c93-4884-b8e5-874050a59c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:16.301058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 874/1762] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:16.412089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:16.413979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:16.568171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb5789-5bd3-4a6a-a02b-99efd4b8534c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:16.573035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a11f26-09c1-4cee-9605-2318c9328007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:16.573902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a11f26-09c1-4cee-9605-2318c9328007 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:16.574339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a11f26-09c1-4cee-9605-2318c9328007 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:16.574766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a11f26-09c1-4cee-9605-2318c9328007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:16.586808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a11f26-09c1-4cee-9605-2318c9328007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:16.588395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a11f26-09c1-4cee-9605-2318c9328007 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:16.601387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 889/1763] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:16.747217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:17.229434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0daeb9-74cb-49fe-9797-4572b12555ff None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:17.229995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0daeb9-74cb-49fe-9797-4572b12555ff None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:17.230096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0daeb9-74cb-49fe-9797-4572b12555ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:17.262926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0f0daeb9-74cb-49fe-9797-4572b12555ff None None] MFA Rules not processed for user `cbcc15c786fb43669ac34a4897735c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:17.267644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0f0daeb9-74cb-49fe-9797-4572b12555ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:17.287126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:17.334265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c7e1-d436-4c65-967a-a777b2303d84 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:17.334893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c7e1-d436-4c65-967a-a777b2303d84 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:17.335520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5c7e1-d436-4c65-967a-a777b2303d84 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:17.337377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 890/1764] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:26:17.352423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a76c3-321b-4717-bbf6-50dad8f61d44 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:17.357250 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a76c3-321b-4717-bbf6-50dad8f61d44 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:17.357250 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a76c3-321b-4717-bbf6-50dad8f61d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:17.436191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-150a76c3-321b-4717-bbf6-50dad8f61d44 None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:17.448004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-150a76c3-321b-4717-bbf6-50dad8f61d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:17.490971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 875/1765] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:17.618851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f14b9-7a8f-480c-84fd-f0a27241adf6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:17.619392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f14b9-7a8f-480c-84fd-f0a27241adf6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:17.619751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f14b9-7a8f-480c-84fd-f0a27241adf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:17.662622 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5d6f14b9-7a8f-480c-84fd-f0a27241adf6 None None] MFA Rules not processed for user `15ff2cacb5c1450482597a2f5fa3c8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:17.664476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5d6f14b9-7a8f-480c-84fd-f0a27241adf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:17.698687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 891/1766] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:17.809608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 876/1767] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:17.965163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a76c3-321b-4717-bbf6-50dad8f61d44 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:17.966633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-150a76c3-321b-4717-bbf6-50dad8f61d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:17.995853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a76c3-321b-4717-bbf6-50dad8f61d44 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:18.002464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:18.005801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:18.006332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:18.006955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] PATH_INFO: `/v3/users/1b775285acd8456e86d679c20955c76d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:18.008961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b775285acd8456e86d679c20955c76d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:18.011917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:18.280263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b775285acd8456e86d679c20955c76d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:18.283967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b775285acd8456e86d679c20955c76d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:18.287619 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b775285acd8456e86d679c20955c76d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:18.315528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b775285acd8456e86d679c20955c76d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:18.359243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b775285acd8456e86d679c20955c76d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:18.500368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 892/1768] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:17 2022] DELETE /identity/v3/users/1b775285acd8456e86d679c20955c76d => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:18.508338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d54519-522f-4a34-840f-8812a9670a65 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:18.508838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d54519-522f-4a34-840f-8812a9670a65 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:18.508913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d54519-522f-4a34-840f-8812a9670a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:18.562861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-10d54519-522f-4a34-840f-8812a9670a65 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:18.564983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-10d54519-522f-4a34-840f-8812a9670a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:18.709101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 877/1769] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:18.787584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:18.789898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:18.811407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d54519-522f-4a34-840f-8812a9670a65 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:18.812830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-10d54519-522f-4a34-840f-8812a9670a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:18.853221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d54519-522f-4a34-840f-8812a9670a65 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:18.882554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:18.883892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:18.884409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:18.884909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] PATH_INFO: `/v3/projects/6d58d32268d64d629caf5f4e7dfca47e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:18.890156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d58d32268d64d629caf5f4e7dfca47e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:18.893183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:18.894460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0adc2-f31f-408a-b5d3-bec074cb92e6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:18.911706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afcc21-6681-4005-90b2-c59f0b7784f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:18.913102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afcc21-6681-4005-90b2-c59f0b7784f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:18.913785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afcc21-6681-4005-90b2-c59f0b7784f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:18.916470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afcc21-6681-4005-90b2-c59f0b7784f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:18.938468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afcc21-6681-4005-90b2-c59f0b7784f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:18.949225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afcc21-6681-4005-90b2-c59f0b7784f5 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:18.970357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 893/1770] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:19.053096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:19.133780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d58d32268d64d629caf5f4e7dfca47e', 'request_id': 'req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.148091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d58d32268d64d629caf5f4e7dfca47e', 'request_id': 'req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.154672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:19.283178 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d58d32268d64d629caf5f4e7dfca47e', 'request_id': 'req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.300157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] The token cache is being invalidate because project 6d58d32268d64d629caf5f4e7dfca47e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:19.300757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.302388 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.305780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 878/1771] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:18 2022] DELETE /identity/v3/projects/6d58d32268d64d629caf5f4e7dfca47e => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:19.309914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afcc21-6681-4005-90b2-c59f0b7784f5 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:19.338017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afcc21-6681-4005-90b2-c59f0b7784f5 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:19.472514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:19.475209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:19.475784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:19.476521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] PATH_INFO: `/v3/users/f05899d20a284ec2adb223dc10331a97` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:19.479082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f05899d20a284ec2adb223dc10331a97)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:19.481921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:19.612176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:19.614001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:19.628669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21a04d-00bb-4d45-9e01-cbc699ad75c2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:19.636079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:19.637297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:19.637857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:19.638337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] PATH_INFO: `/v3/users/9d1535cb30a547358ec7197185ffd67d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:19.640236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1535cb30a547358ec7197185ffd67d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:19.643736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:19.712047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:19.785385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f05899d20a284ec2adb223dc10331a97'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.786181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f05899d20a284ec2adb223dc10331a97'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.786989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f05899d20a284ec2adb223dc10331a97'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.801924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f05899d20a284ec2adb223dc10331a97'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.837343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-53acf1d8-f0ab-4129-9c13-d91bf01edb35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f05899d20a284ec2adb223dc10331a97'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.888347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1535cb30a547358ec7197185ffd67d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.889533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1535cb30a547358ec7197185ffd67d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.890998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1535cb30a547358ec7197185ffd67d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.922043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1535cb30a547358ec7197185ffd67d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:19.953261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1535cb30a547358ec7197185ffd67d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:20.009268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 894/1772] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:19 2022] DELETE /identity/v3/users/f05899d20a284ec2adb223dc10331a97 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:20.113661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 879/1773] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:19 2022] DELETE /identity/v3/users/9d1535cb30a547358ec7197185ffd67d => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:26:20.115445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f894a-6f3f-465a-9ee2-753c4e39718b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:20.115741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f894a-6f3f-465a-9ee2-753c4e39718b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:20.116177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f894a-6f3f-465a-9ee2-753c4e39718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:20.136106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-692f894a-6f3f-465a-9ee2-753c4e39718b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:20.146216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-692f894a-6f3f-465a-9ee2-753c4e39718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:20.320889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 895/1774] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:20.331341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:20.333801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:20.405803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4b942-3fca-4385-a70c-7b679d97574c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:20.411051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:20.411945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:20.412232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:20.412573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:20.443154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f894a-6f3f-465a-9ee2-753c4e39718b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:20.450425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-692f894a-6f3f-465a-9ee2-753c4e39718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:20.488601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:20.490761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:20.514670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 880/1775] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:20.714233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f894a-6f3f-465a-9ee2-753c4e39718b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:20.736563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:20.736563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:20.736563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:20.736563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:20.742033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:20.760909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:20.765297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:20.780225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 896/1776] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:26:21.073934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:21.076820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:21.097670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da3c32-45cf-4b34-b75f-09a596307ea2 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:21.107944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:21.109137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:21.111344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:21.111344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] PATH_INFO: `/v3/projects/b8eec917728548c4a830cafd4adab801` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:21.123765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8eec917728548c4a830cafd4adab801)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:21.129355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:21.442938 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:21.445250 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:21.480990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb59fe-9b22-41c8-8f1b-d0866cded206 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:21.485506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8eec917728548c4a830cafd4adab801', 'request_id': 'req-af811805-c0d4-461c-831f-6559c2d7285f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:21.509032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8eec917728548c4a830cafd4adab801', 'request_id': 'req-af811805-c0d4-461c-831f-6559c2d7285f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:21.543997 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:21.620139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:21.621268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:21.621623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:21.622173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] PATH_INFO: `/v3/projects/e360c94bd98b442aae832d2542c9cc60` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:21.624418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e360c94bd98b442aae832d2542c9cc60)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:21.627185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:21.629468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8eec917728548c4a830cafd4adab801', 'request_id': 'req-af811805-c0d4-461c-831f-6559c2d7285f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:21.669975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] The token cache is being invalidate because project b8eec917728548c4a830cafd4adab801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:21.670436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:21.672278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-af811805-c0d4-461c-831f-6559c2d7285f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:21.692707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 881/1777] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:21 2022] DELETE /identity/v3/projects/b8eec917728548c4a830cafd4adab801 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:21.696514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6c9e7-6af5-432f-9cd5-f9876ecd65d4 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:21.699533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6c9e7-6af5-432f-9cd5-f9876ecd65d4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:21.700749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6c9e7-6af5-432f-9cd5-f9876ecd65d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:21.757325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0da6c9e7-6af5-432f-9cd5-f9876ecd65d4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:21.758949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0da6c9e7-6af5-432f-9cd5-f9876ecd65d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:21.769610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 882/1778] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:21.787064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0185-e15f-49bb-9b67-31961a1f2760 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:21.787064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0185-e15f-49bb-9b67-31961a1f2760 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:21.787064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d0185-e15f-49bb-9b67-31961a1f2760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:21.841102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-446d0185-e15f-49bb-9b67-31961a1f2760 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:21.842803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-446d0185-e15f-49bb-9b67-31961a1f2760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:21.844795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e360c94bd98b442aae832d2542c9cc60', 'request_id': 'req-eca545c0-ddda-407d-9997-59851fcd7df1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:21.855449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e360c94bd98b442aae832d2542c9cc60', 'request_id': 'req-eca545c0-ddda-407d-9997-59851fcd7df1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:21.967682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 883/1779] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:21.968826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d0185-e15f-49bb-9b67-31961a1f2760 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:21.985990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d0185-e15f-49bb-9b67-31961a1f2760 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:21.991644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:21.992350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:21.992642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:21.992961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:21.993801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:21.994177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:21.996190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.009600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 884/1780] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:22.018475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.030728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bae091-7bab-448c-8a22-4032f8162311 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.037820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.038534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.038850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.039174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.040036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.042270 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.154778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e360c94bd98b442aae832d2542c9cc60', 'request_id': 'req-eca545c0-ddda-407d-9997-59851fcd7df1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:22.167739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] The token cache is being invalidate because project e360c94bd98b442aae832d2542c9cc60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:22.168034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:22.168692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-eca545c0-ddda-407d-9997-59851fcd7df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:22.170496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 897/1781] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:21 2022] DELETE /identity/v3/projects/e360c94bd98b442aae832d2542c9cc60 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:22.175988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad91a99-7292-42b6-8013-e6c5af638865 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.176560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad91a99-7292-42b6-8013-e6c5af638865 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.176810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad91a99-7292-42b6-8013-e6c5af638865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.195176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1ad91a99-7292-42b6-8013-e6c5af638865 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:22.196418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ad91a99-7292-42b6-8013-e6c5af638865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:22.205806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 898/1782] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:22.214604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1c44-f024-4906-b57c-ea65fc03b2a6 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.214891 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1c44-f024-4906-b57c-ea65fc03b2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.215169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d1c44-f024-4906-b57c-ea65fc03b2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.245388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-396d1c44-f024-4906-b57c-ea65fc03b2a6 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:22.247232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-396d1c44-f024-4906-b57c-ea65fc03b2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:22.264404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 899/1783] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:22.265932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d1c44-f024-4906-b57c-ea65fc03b2a6 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.283534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 885/1784] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:22 2022] POST /identity/v3/projects => generated 381 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:22.285057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.288269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d1c44-f024-4906-b57c-ea65fc03b2a6 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.288821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:22.396429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9168644-064a-401e-9a63-5492a9d069dc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.402348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.403732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.404245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.408489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] PATH_INFO: `/v3/projects/16cb671b750e4052aa2b2bdb3d8a26d9` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.408489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] RBAC: Authorizing `identity:delete_project(project_id=16cb671b750e4052aa2b2bdb3d8a26d9)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.408489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.445383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.445789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.447512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.454530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.454530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:22.454530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.454530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.467021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 900/1785] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:22.468213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.471519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:22.486434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75823fc-a250-4ae8-9927-a38404e3213e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.501940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a306-9a16-413c-b7ae-4969a5ce0b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.503042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a306-9a16-413c-b7ae-4969a5ce0b33 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.503777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a306-9a16-413c-b7ae-4969a5ce0b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.503777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961a306-9a16-413c-b7ae-4969a5ce0b33 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.504620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961a306-9a16-413c-b7ae-4969a5ce0b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.506985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961a306-9a16-413c-b7ae-4969a5ce0b33 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.564350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16cb671b750e4052aa2b2bdb3d8a26d9', 'request_id': 'req-32711feb-82bb-4aac-a723-825d24ab507f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:22.569910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16cb671b750e4052aa2b2bdb3d8a26d9', 'request_id': 'req-32711feb-82bb-4aac-a723-825d24ab507f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:22.583692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:22.644720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 901/1786] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:22 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:22.644720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a306-9a16-413c-b7ae-4969a5ce0b33 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.655865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961a306-9a16-413c-b7ae-4969a5ce0b33 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.674748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16cb671b750e4052aa2b2bdb3d8a26d9', 'request_id': 'req-32711feb-82bb-4aac-a723-825d24ab507f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:22.687328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] The token cache is being invalidate because project 16cb671b750e4052aa2b2bdb3d8a26d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:22.687623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:22.688299 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:22.689932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 886/1787] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:22 2022] DELETE /identity/v3/projects/16cb671b750e4052aa2b2bdb3d8a26d9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:22.690764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.692158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:22.722547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c7ed4-c49f-487f-9fbf-ba3ad251b152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.723994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c7ed4-c49f-487f-9fbf-ba3ad251b152 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.723994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c7ed4-c49f-487f-9fbf-ba3ad251b152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.723994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c7ed4-c49f-487f-9fbf-ba3ad251b152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.724757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c7ed4-c49f-487f-9fbf-ba3ad251b152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.726863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c7ed4-c49f-487f-9fbf-ba3ad251b152 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.763833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32711feb-82bb-4aac-a723-825d24ab507f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.768773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.769583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.769873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.770150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.770886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.772568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.785192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 887/1788] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:22.786766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.787997 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:22.805285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a751bf-bdcb-4403-8a29-bfc029c1be61 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.810232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.811121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.811441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.811724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] PATH_INFO: `/v3/users/de02739fb4d94df0a48293b7c22586b6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.812948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de02739fb4d94df0a48293b7c22586b6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.814925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.832802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 902/1789] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:22 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:22.833675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c7ed4-c49f-487f-9fbf-ba3ad251b152 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.915513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c7ed4-c49f-487f-9fbf-ba3ad251b152 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.920391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.921064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.921359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.921668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.922390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.924053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:22.934049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 903/1790] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:22.934825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:22.936487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:22.949183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de0eb9-4070-4e72-a172-69d41d540f10 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:22.953775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:22.954960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:22.955269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:22.955555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:22.956238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:22.958139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.071584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de02739fb4d94df0a48293b7c22586b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:23.072254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de02739fb4d94df0a48293b7c22586b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:23.072870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de02739fb4d94df0a48293b7c22586b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:23.080443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de02739fb4d94df0a48293b7c22586b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:23.092476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de02739fb4d94df0a48293b7c22586b6'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:23.095554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 904/1791] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:22 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:23.095554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.096104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:23.108894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c18ad-f049-4d36-a04d-e332d0e06efd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.113808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc630e6c-a8d8-4e6e-8458-a8b71a8109fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.114698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc630e6c-a8d8-4e6e-8458-a8b71a8109fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.114989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc630e6c-a8d8-4e6e-8458-a8b71a8109fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.115295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc630e6c-a8d8-4e6e-8458-a8b71a8109fe admin admin] PATH_INFO: `/v3/projects/45c653038a29408e929aa46078d89493/users/147fa7a0291341b081cca557740226a7/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.130342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc630e6c-a8d8-4e6e-8458-a8b71a8109fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=45c653038a29408e929aa46078d89493, user_id=147fa7a0291341b081cca557740226a7, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.133040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc630e6c-a8d8-4e6e-8458-a8b71a8109fe admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.189498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 888/1792] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:22 2022] DELETE /identity/v3/users/de02739fb4d94df0a48293b7c22586b6 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:23.190397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.191888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:23.252667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 905/1793] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:23 2022] PUT /identity/v3/projects/45c653038a29408e929aa46078d89493/users/147fa7a0291341b081cca557740226a7/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:23.263012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffc59f-0eb6-4032-aab6-e9963c97f056 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.263330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffc59f-0eb6-4032-aab6-e9963c97f056 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.263611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffc59f-0eb6-4032-aab6-e9963c97f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.266330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff100857-5e51-47b1-bae2-d9f02c26bcd6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.306179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-81ffc59f-0eb6-4032-aab6-e9963c97f056 None None] MFA Rules not processed for user `147fa7a0291341b081cca557740226a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:23.307535 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-81ffc59f-0eb6-4032-aab6-e9963c97f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:23.325649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84cb4e-2777-4c10-82c3-c35adf6c1459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.326326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84cb4e-2777-4c10-82c3-c35adf6c1459 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.326604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84cb4e-2777-4c10-82c3-c35adf6c1459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.326879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84cb4e-2777-4c10-82c3-c35adf6c1459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.327753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84cb4e-2777-4c10-82c3-c35adf6c1459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.329374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84cb4e-2777-4c10-82c3-c35adf6c1459 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.341720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 889/1794] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:23.346073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84cb4e-2777-4c10-82c3-c35adf6c1459 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.352588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84cb4e-2777-4c10-82c3-c35adf6c1459 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.357380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6eb4b-7049-4271-ba4b-e820b156b7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.358058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6eb4b-7049-4271-ba4b-e820b156b7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.358337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6eb4b-7049-4271-ba4b-e820b156b7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.358615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6eb4b-7049-4271-ba4b-e820b156b7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.359362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6eb4b-7049-4271-ba4b-e820b156b7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.361459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6eb4b-7049-4271-ba4b-e820b156b7b5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.375164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 890/1795] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:26:23.376076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 906/1796] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:23.376623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6eb4b-7049-4271-ba4b-e820b156b7b5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.380378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffc59f-0eb6-4032-aab6-e9963c97f056 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.386373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6eb4b-7049-4271-ba4b-e820b156b7b5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.390209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffc59f-0eb6-4032-aab6-e9963c97f056 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.391952 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5e2e3-36c4-4011-9f0b-2e6a46600c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.392386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e2e3-36c4-4011-9f0b-2e6a46600c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.392667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e2e3-36c4-4011-9f0b-2e6a46600c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.392940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e2e3-36c4-4011-9f0b-2e6a46600c47 admin admin] PATH_INFO: `/v3/projects/ad0d6fc02c7b4376ad7c43f2a1f710dd/users/d74e2c2e492e496f91bc01e6a4c070dc/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.395705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2f437-5705-4d2c-9731-3565b5c497b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.396359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f437-5705-4d2c-9731-3565b5c497b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.396646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f437-5705-4d2c-9731-3565b5c497b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.396919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2f437-5705-4d2c-9731-3565b5c497b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.397711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2f437-5705-4d2c-9731-3565b5c497b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.399789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2f437-5705-4d2c-9731-3565b5c497b4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.407894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5e2e3-36c4-4011-9f0b-2e6a46600c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0d6fc02c7b4376ad7c43f2a1f710dd, user_id=d74e2c2e492e496f91bc01e6a4c070dc, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.410574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5e2e3-36c4-4011-9f0b-2e6a46600c47 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.511410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 891/1797] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:23 2022] PUT /identity/v3/projects/ad0d6fc02c7b4376ad7c43f2a1f710dd/users/d74e2c2e492e496f91bc01e6a4c070dc/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:23.522098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159500b-3b7e-4d16-8e71-84dfa6a43781 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.522474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159500b-3b7e-4d16-8e71-84dfa6a43781 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.522826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159500b-3b7e-4d16-8e71-84dfa6a43781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.550556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 907/1798] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:23 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:23.551876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2f437-5705-4d2c-9731-3565b5c497b4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.561894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2159500b-3b7e-4d16-8e71-84dfa6a43781 None None] MFA Rules not processed for user `d74e2c2e492e496f91bc01e6a4c070dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:23.564399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2159500b-3b7e-4d16-8e71-84dfa6a43781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:23.645350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 892/1799] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:23.649187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159500b-3b7e-4d16-8e71-84dfa6a43781 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.659891 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2f437-5705-4d2c-9731-3565b5c497b4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.666713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.667402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159500b-3b7e-4d16-8e71-84dfa6a43781 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.667799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.678221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.678221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.678221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.678221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.678610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0100-586b-4293-9f1b-af92010676e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.678610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0100-586b-4293-9f1b-af92010676e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.678994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0100-586b-4293-9f1b-af92010676e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.679479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0100-586b-4293-9f1b-af92010676e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.680379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0100-586b-4293-9f1b-af92010676e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.682395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0100-586b-4293-9f1b-af92010676e6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.822014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 908/1800] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:23 2022] POST /identity/v3/users => generated 298 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:23.822835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.824693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:23.840426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7631019-35e5-4be9-9d39-bd53d5028f52 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.845849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.853157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.853157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.853157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] PATH_INFO: `/v3/projects/d7f3c17f45ef434492e19fad56219d7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.853157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f3c17f45ef434492e19fad56219d7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.853157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.858176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 893/1801] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:23 2022] POST /identity/v3/projects => generated 387 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:23.859566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0100-586b-4293-9f1b-af92010676e6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.942493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0100-586b-4293-9f1b-af92010676e6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.950363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f495e3-b9e9-40d4-974b-12ee96a563e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.951944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f495e3-b9e9-40d4-974b-12ee96a563e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.952458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f495e3-b9e9-40d4-974b-12ee96a563e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.953488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f495e3-b9e9-40d4-974b-12ee96a563e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.954517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f495e3-b9e9-40d4-974b-12ee96a563e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.957480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f495e3-b9e9-40d4-974b-12ee96a563e1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:23.971094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 894/1802] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:23.972076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f495e3-b9e9-40d4-974b-12ee96a563e1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:23.981809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f495e3-b9e9-40d4-974b-12ee96a563e1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:23.986651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bdfc5-e047-4e30-a0b7-adb5276f0c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:23.987367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bdfc5-e047-4e30-a0b7-adb5276f0c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:23.987662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bdfc5-e047-4e30-a0b7-adb5276f0c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:23.987944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94bdfc5-e047-4e30-a0b7-adb5276f0c50 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:23.988639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bdfc5-e047-4e30-a0b7-adb5276f0c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:23.990557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94bdfc5-e047-4e30-a0b7-adb5276f0c50 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.005189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f3c17f45ef434492e19fad56219d7f', 'request_id': 'req-d750210a-08d8-4e3a-9d8b-109a753e1ee6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:24.011850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f3c17f45ef434492e19fad56219d7f', 'request_id': 'req-d750210a-08d8-4e3a-9d8b-109a753e1ee6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:24.121208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 895/1803] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:23 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:24.122180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bdfc5-e047-4e30-a0b7-adb5276f0c50 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.132275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f3c17f45ef434492e19fad56219d7f', 'request_id': 'req-d750210a-08d8-4e3a-9d8b-109a753e1ee6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:24.144900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94bdfc5-e047-4e30-a0b7-adb5276f0c50 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.147684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] The token cache is being invalidate because project d7f3c17f45ef434492e19fad56219d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:24.147984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:24.148651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:24.150227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 909/1804] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:23 2022] DELETE /identity/v3/projects/d7f3c17f45ef434492e19fad56219d7f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:24.151040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.152356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:24.217587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7841be-a3a6-48e2-8ab2-5bbfbda6fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.218566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7841be-a3a6-48e2-8ab2-5bbfbda6fc9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.218853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7841be-a3a6-48e2-8ab2-5bbfbda6fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.219130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7841be-a3a6-48e2-8ab2-5bbfbda6fc9a admin admin] PATH_INFO: `/v3/projects/7f088e9b5cf048b08136abf8574d804b/users/f69da77f50c940a482f5d43d9341e8c5/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.231354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750210a-08d8-4e3a-9d8b-109a753e1ee6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.233757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7841be-a3a6-48e2-8ab2-5bbfbda6fc9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f088e9b5cf048b08136abf8574d804b, user_id=f69da77f50c940a482f5d43d9341e8c5, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.237252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7841be-a3a6-48e2-8ab2-5bbfbda6fc9a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.237652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec68e3b-1ffd-44b9-a9fb-12768cffafc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.238323 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68e3b-1ffd-44b9-a9fb-12768cffafc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.238743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68e3b-1ffd-44b9-a9fb-12768cffafc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.239058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68e3b-1ffd-44b9-a9fb-12768cffafc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.239989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec68e3b-1ffd-44b9-a9fb-12768cffafc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.241889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec68e3b-1ffd-44b9-a9fb-12768cffafc4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.253298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 910/1805] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:24.254852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5caec-58bd-4609-a5e2-0d2d058cb01e None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.256871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5caec-58bd-4609-a5e2-0d2d058cb01e None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.256871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5caec-58bd-4609-a5e2-0d2d058cb01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.273646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8bf5caec-58bd-4609-a5e2-0d2d058cb01e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:24.274848 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8bf5caec-58bd-4609-a5e2-0d2d058cb01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:24.283838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 911/1806] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:24.284688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5caec-58bd-4609-a5e2-0d2d058cb01e None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.295572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:24.296185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5caec-58bd-4609-a5e2-0d2d058cb01e None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.302713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 896/1807] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:24 2022] PUT /identity/v3/projects/7f088e9b5cf048b08136abf8574d804b/users/f69da77f50c940a482f5d43d9341e8c5/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:24.308676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351f56-b1bc-4d62-9ece-407c7fe7b55c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.308676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351f56-b1bc-4d62-9ece-407c7fe7b55c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.308676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351f56-b1bc-4d62-9ece-407c7fe7b55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.324267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-59351f56-b1bc-4d62-9ece-407c7fe7b55c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:24.325629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-59351f56-b1bc-4d62-9ece-407c7fe7b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:24.366696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2feee2-5dd2-45e0-993a-3f7cc2cae9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.367631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2feee2-5dd2-45e0-993a-3f7cc2cae9c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.367947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2feee2-5dd2-45e0-993a-3f7cc2cae9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.368267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2feee2-5dd2-45e0-993a-3f7cc2cae9c3 admin admin] PATH_INFO: `/v3/projects/50b0ecc0be434c5e966f00905ae89fda/users/164c17d09d624a64be6211fc6b4b65e7/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.383997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2feee2-5dd2-45e0-993a-3f7cc2cae9c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b0ecc0be434c5e966f00905ae89fda, user_id=164c17d09d624a64be6211fc6b4b65e7, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.386307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 897/1808] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:24.386767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2feee2-5dd2-45e0-993a-3f7cc2cae9c3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.387175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59351f56-b1bc-4d62-9ece-407c7fe7b55c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.388486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-59351f56-b1bc-4d62-9ece-407c7fe7b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:24.400591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59351f56-b1bc-4d62-9ece-407c7fe7b55c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.405154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b2405-c727-42fe-80bb-9b0bb395baa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.405863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2405-c727-42fe-80bb-9b0bb395baa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.406149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2405-c727-42fe-80bb-9b0bb395baa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.406427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b2405-c727-42fe-80bb-9b0bb395baa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.407187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b2405-c727-42fe-80bb-9b0bb395baa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.408796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b2405-c727-42fe-80bb-9b0bb395baa6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.418553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 898/1809] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:24.419378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b2405-c727-42fe-80bb-9b0bb395baa6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.429443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b2405-c727-42fe-80bb-9b0bb395baa6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.434409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.435244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.435607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.435882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.436581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:24.436885 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.438946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.450947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 899/1810] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:24.451874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.458276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 912/1811] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:24 2022] PUT /identity/v3/projects/50b0ecc0be434c5e966f00905ae89fda/users/164c17d09d624a64be6211fc6b4b65e7/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:24.459243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2feee2-5dd2-45e0-993a-3f7cc2cae9c3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.463713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad7fd4-d918-4eca-a756-7956bb88da47 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.520280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2feee2-5dd2-45e0-993a-3f7cc2cae9c3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.520491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e970c-b396-48bc-82d1-7ea5bd3e8105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.521254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e970c-b396-48bc-82d1-7ea5bd3e8105 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.521542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e970c-b396-48bc-82d1-7ea5bd3e8105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.521819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e970c-b396-48bc-82d1-7ea5bd3e8105 admin admin] PATH_INFO: `/v3/projects/7f088e9b5cf048b08136abf8574d804b/users/f69da77f50c940a482f5d43d9341e8c5/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.523911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e970c-b396-48bc-82d1-7ea5bd3e8105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f088e9b5cf048b08136abf8574d804b, user_id=f69da77f50c940a482f5d43d9341e8c5, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.525028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b0d30-e0f4-47c3-ac88-2e7de1e534a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.525691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0d30-e0f4-47c3-ac88-2e7de1e534a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.526605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0d30-e0f4-47c3-ac88-2e7de1e534a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.526605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b0d30-e0f4-47c3-ac88-2e7de1e534a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.526765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e970c-b396-48bc-82d1-7ea5bd3e8105 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.526973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b0d30-e0f4-47c3-ac88-2e7de1e534a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.528960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b0d30-e0f4-47c3-ac88-2e7de1e534a2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.536694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 900/1812] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:24 2022] PUT /identity/v3/projects/7f088e9b5cf048b08136abf8574d804b/users/f69da77f50c940a482f5d43d9341e8c5/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:24.546344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e970c-b396-48bc-82d1-7ea5bd3e8105 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.571158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:24.597574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e970c-b396-48bc-82d1-7ea5bd3e8105 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.602993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2d7f-2c5f-44a6-acae-2d4202e9b79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.603727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2d7f-2c5f-44a6-acae-2d4202e9b79b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.604013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2d7f-2c5f-44a6-acae-2d4202e9b79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.604289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2d7f-2c5f-44a6-acae-2d4202e9b79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.605026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f2d7f-2c5f-44a6-acae-2d4202e9b79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.606633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f2d7f-2c5f-44a6-acae-2d4202e9b79b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.607246 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 913/1813] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:24 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:24.609272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb0067-6262-4be8-82f4-39de431330b2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.609569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb0067-6262-4be8-82f4-39de431330b2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.610947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb0067-6262-4be8-82f4-39de431330b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.617712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 901/1814] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:24.620166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2d7f-2c5f-44a6-acae-2d4202e9b79b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.635861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2d7f-2c5f-44a6-acae-2d4202e9b79b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.640372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3afb0067-6262-4be8-82f4-39de431330b2 None None] MFA Rules not processed for user `f69da77f50c940a482f5d43d9341e8c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:24.642170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3afb0067-6262-4be8-82f4-39de431330b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:24.713094 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2a5dc-0c2f-4e24-8f85-613d4ff30b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.714108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a5dc-0c2f-4e24-8f85-613d4ff30b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.714578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a5dc-0c2f-4e24-8f85-613d4ff30b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.715101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a5dc-0c2f-4e24-8f85-613d4ff30b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.715989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2a5dc-0c2f-4e24-8f85-613d4ff30b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.716780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 914/1815] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:24.717497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb0067-6262-4be8-82f4-39de431330b2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.718349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2a5dc-0c2f-4e24-8f85-613d4ff30b2b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.734823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb0067-6262-4be8-82f4-39de431330b2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.751470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.751470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.751470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.751470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] PATH_INFO: `/v3/projects/50b0ecc0be434c5e966f00905ae89fda/users/164c17d09d624a64be6211fc6b4b65e7/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.752152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b0ecc0be434c5e966f00905ae89fda, user_id=164c17d09d624a64be6211fc6b4b65e7, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.752152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.757799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 915/1816] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:24 2022] PUT /identity/v3/projects/50b0ecc0be434c5e966f00905ae89fda/users/164c17d09d624a64be6211fc6b4b65e7/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:24.760126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.761855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:24.821829 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983951c-fc45-4179-8e3b-fdfe5121a6ca admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:24.825755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719aa1b-ccbc-4be6-8e6d-84ee85e12d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:24.826483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719aa1b-ccbc-4be6-8e6d-84ee85e12d1f service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.826788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719aa1b-ccbc-4be6-8e6d-84ee85e12d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.827070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719aa1b-ccbc-4be6-8e6d-84ee85e12d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.888185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 902/1817] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:24 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:24.889583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7eafd-2890-4b14-8784-94e5be71d8e6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:24.889855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7eafd-2890-4b14-8784-94e5be71d8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:24.890139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7eafd-2890-4b14-8784-94e5be71d8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:24.931383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719aa1b-ccbc-4be6-8e6d-84ee85e12d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:24.933004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719aa1b-ccbc-4be6-8e6d-84ee85e12d1f service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:24.948705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 916/1818] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:24.949795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719aa1b-ccbc-4be6-8e6d-84ee85e12d1f service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:24.957493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a5e7eafd-2890-4b14-8784-94e5be71d8e6 None None] MFA Rules not processed for user `164c17d09d624a64be6211fc6b4b65e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:24.959109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a5e7eafd-2890-4b14-8784-94e5be71d8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:25.094915 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719aa1b-ccbc-4be6-8e6d-84ee85e12d1f service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:25.109335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539c6fd-34cb-4de5-90d6-0e8ddd961d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:25.120159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539c6fd-34cb-4de5-90d6-0e8ddd961d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.120159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539c6fd-34cb-4de5-90d6-0e8ddd961d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.120159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539c6fd-34cb-4de5-90d6-0e8ddd961d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.120159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539c6fd-34cb-4de5-90d6-0e8ddd961d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:25.120159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4539c6fd-34cb-4de5-90d6-0e8ddd961d1e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:25.125377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 917/1819] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:25.126341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539c6fd-34cb-4de5-90d6-0e8ddd961d1e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:25.143288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 903/1820] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:25.144318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7eafd-2890-4b14-8784-94e5be71d8e6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:25.147865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539c6fd-34cb-4de5-90d6-0e8ddd961d1e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:25.158319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:25.159050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.159425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.159764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.162087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:25.163642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7eafd-2890-4b14-8784-94e5be71d8e6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:25.169370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:25.169906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.170290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.170663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.171551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:25.173942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:25.201730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 904/1821] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:25.203036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:25.205272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:25.217677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f63401-b58c-457f-b85c-28eb477775ee admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:25.222508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e310-4661-4585-994e-5179fc01623d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:25.223160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e310-4661-4585-994e-5179fc01623d service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.223537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e310-4661-4585-994e-5179fc01623d service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.223821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e310-4661-4585-994e-5179fc01623d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.234464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704e310-4661-4585-994e-5179fc01623d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:25.237452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704e310-4661-4585-994e-5179fc01623d service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:25.246830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:25.248601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:25.249099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 905/1822] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:25.249971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e310-4661-4585-994e-5179fc01623d service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:25.263660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e310-4661-4585-994e-5179fc01623d service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:25.267464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 918/1823] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:25.271332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c40f98-4a57-4961-933f-981e39c74191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:25.272662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c40f98-4a57-4961-933f-981e39c74191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.272952 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c40f98-4a57-4961-933f-981e39c74191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.273393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c40f98-4a57-4961-933f-981e39c74191 admin admin] PATH_INFO: `/v3/projects/80913cca9b594267870c36e4a7308407/users/134a94416f12451eb07942b8a797c35d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.286110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:25.291452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c40f98-4a57-4961-933f-981e39c74191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80913cca9b594267870c36e4a7308407, user_id=134a94416f12451eb07942b8a797c35d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:25.294375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c40f98-4a57-4961-933f-981e39c74191 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:25.298044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af365b-932d-4c5e-ab64-9071ee4ba176 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:25.302960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:25.303992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.304285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.304571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] PATH_INFO: `/v3/users/d74e2c2e492e496f91bc01e6a4c070dc` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.305958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74e2c2e492e496f91bc01e6a4c070dc)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:25.308044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:25.401854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 906/1824] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:25 2022] PUT /identity/v3/projects/80913cca9b594267870c36e4a7308407/users/134a94416f12451eb07942b8a797c35d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:25.412303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a64db5-6839-432e-8f67-66730eac6594 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.412624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a64db5-6839-432e-8f67-66730eac6594 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.412905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a64db5-6839-432e-8f67-66730eac6594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.442889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-81a64db5-6839-432e-8f67-66730eac6594 None None] MFA Rules not processed for user `134a94416f12451eb07942b8a797c35d`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:25.444254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-81a64db5-6839-432e-8f67-66730eac6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:25.513209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 907/1825] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:25.514266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a64db5-6839-432e-8f67-66730eac6594 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:25.543919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74e2c2e492e496f91bc01e6a4c070dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:25.544587 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74e2c2e492e496f91bc01e6a4c070dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:25.545199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74e2c2e492e496f91bc01e6a4c070dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:25.553807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74e2c2e492e496f91bc01e6a4c070dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:25.567172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1dff2384-08a9-4dde-a8ec-61b2bf2fe627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74e2c2e492e496f91bc01e6a4c070dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:25.586595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a64db5-6839-432e-8f67-66730eac6594 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:25.591822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:25.592534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.592829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.593238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.593955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:25.595937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:25.651990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 919/1826] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:25 2022] DELETE /identity/v3/users/d74e2c2e492e496f91bc01e6a4c070dc => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:25.656896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4236c-f88e-4f9a-9dde-5a83761c2a18 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:25.657808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4236c-f88e-4f9a-9dde-5a83761c2a18 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:25.658157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4236c-f88e-4f9a-9dde-5a83761c2a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:25.684907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8fe4236c-f88e-4f9a-9dde-5a83761c2a18 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:25.687846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8fe4236c-f88e-4f9a-9dde-5a83761c2a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:25.773964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 920/1827] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:25.785914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4236c-f88e-4f9a-9dde-5a83761c2a18 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:25.787437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8fe4236c-f88e-4f9a-9dde-5a83761c2a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:25.801618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 908/1828] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:25 2022] POST /identity/v3/projects => generated 383 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:25.802940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:25.805467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:25.857003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:25.918624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4236c-f88e-4f9a-9dde-5a83761c2a18 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:25.998960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefab69-8f8e-46e7-bfa9-2fdba608c1a7 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:26.007223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e64b9-43fe-465d-ab3e-8749ba734150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:26.007973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e64b9-43fe-465d-ab3e-8749ba734150 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:26.013800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e64b9-43fe-465d-ab3e-8749ba734150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:26.013800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e64b9-43fe-465d-ab3e-8749ba734150 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:26.013800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e64b9-43fe-465d-ab3e-8749ba734150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:26.013800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e64b9-43fe-465d-ab3e-8749ba734150 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:26.054740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468c02-41f0-4b83-957e-3c8d82fdee65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:26.054740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468c02-41f0-4b83-957e-3c8d82fdee65 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:26.054740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468c02-41f0-4b83-957e-3c8d82fdee65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:26.054740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5468c02-41f0-4b83-957e-3c8d82fdee65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:26.060117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5468c02-41f0-4b83-957e-3c8d82fdee65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:26.061833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5468c02-41f0-4b83-957e-3c8d82fdee65 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:26.084832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 921/1829] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:26.158087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 909/1830] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:25 2022] POST /identity/v3/users => generated 298 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:26.160558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468c02-41f0-4b83-957e-3c8d82fdee65 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:26.175378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5468c02-41f0-4b83-957e-3c8d82fdee65 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:26.191532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838693ec-688f-40fb-b92f-1c649e37c526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:26.191759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838693ec-688f-40fb-b92f-1c649e37c526 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:26.192041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838693ec-688f-40fb-b92f-1c649e37c526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:26.192312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838693ec-688f-40fb-b92f-1c649e37c526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:26.194570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838693ec-688f-40fb-b92f-1c649e37c526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:26.195516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838693ec-688f-40fb-b92f-1c649e37c526 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:26.231143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 922/1831] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:26.234839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e64b9-43fe-465d-ab3e-8749ba734150 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:26.252055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e64b9-43fe-465d-ab3e-8749ba734150 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:26.261533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84add9a4-c68b-46be-a84b-eb6ddafc98df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:26.262453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add9a4-c68b-46be-a84b-eb6ddafc98df admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:26.262744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add9a4-c68b-46be-a84b-eb6ddafc98df admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:26.263026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add9a4-c68b-46be-a84b-eb6ddafc98df admin admin] PATH_INFO: `/v3/projects/6191a563a555474e9d838929bddcb073/users/c4ccdd588f264ec0ba080878e2b3d359/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:26.288887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add9a4-c68b-46be-a84b-eb6ddafc98df admin admin] RBAC: Authorizing `identity:create_grant(project_id=6191a563a555474e9d838929bddcb073, user_id=c4ccdd588f264ec0ba080878e2b3d359, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:26.292021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add9a4-c68b-46be-a84b-eb6ddafc98df admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:26.389452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:26.451638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 910/1832] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:26 2022] PUT /identity/v3/projects/6191a563a555474e9d838929bddcb073/users/c4ccdd588f264ec0ba080878e2b3d359/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:26:26.456202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838693ec-688f-40fb-b92f-1c649e37c526 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:26.468876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838693ec-688f-40fb-b92f-1c649e37c526 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:26.529758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac7330-b13e-4192-80e6-d61efe264e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:26.537688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac7330-b13e-4192-80e6-d61efe264e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:26.541132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac7330-b13e-4192-80e6-d61efe264e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:26.541132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac7330-b13e-4192-80e6-d61efe264e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:26.541132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac7330-b13e-4192-80e6-d61efe264e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:26.541132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac7330-b13e-4192-80e6-d61efe264e31 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:26.552634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 923/1833] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:26:26.556666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84add9a4-c68b-46be-a84b-eb6ddafc98df admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:26.568955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84add9a4-c68b-46be-a84b-eb6ddafc98df admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:26.574120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:26.575023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:26.575343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:26.575675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] PATH_INFO: `/v3/projects/6191a563a555474e9d838929bddcb073/users/c4ccdd588f264ec0ba080878e2b3d359/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:26.577649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6191a563a555474e9d838929bddcb073, user_id=c4ccdd588f264ec0ba080878e2b3d359, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:26.580370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:26.591145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 911/1834] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:26 2022] PUT /identity/v3/projects/6191a563a555474e9d838929bddcb073/users/c4ccdd588f264ec0ba080878e2b3d359/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:26.603138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0522de-7746-4be3-93df-1f4289c1e375 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:26.603555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0522de-7746-4be3-93df-1f4289c1e375 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:26.603908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0522de-7746-4be3-93df-1f4289c1e375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:26.637607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ef0522de-7746-4be3-93df-1f4289c1e375 None None] MFA Rules not processed for user `c4ccdd588f264ec0ba080878e2b3d359`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:26.639930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ef0522de-7746-4be3-93df-1f4289c1e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:26.762092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 924/1835] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:26.774278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:26.776044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:26.837523 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ca180-126d-4e21-8383-07b968a3ee0f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:26.841116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90142d-c6f9-4261-a117-c0dd5054b2a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:26.841770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90142d-c6f9-4261-a117-c0dd5054b2a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:26.842055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90142d-c6f9-4261-a117-c0dd5054b2a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:26.842333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90142d-c6f9-4261-a117-c0dd5054b2a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:26.844163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7e90142d-c6f9-4261-a117-c0dd5054b2a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:26.917238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90142d-c6f9-4261-a117-c0dd5054b2a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:26.918857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e90142d-c6f9-4261-a117-c0dd5054b2a4 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:26.930965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 912/1836] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:27.134724 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0522de-7746-4be3-93df-1f4289c1e375 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:27.136705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ef0522de-7746-4be3-93df-1f4289c1e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:27.246433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0522de-7746-4be3-93df-1f4289c1e375 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:27.252080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:27.253464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:27.253879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:27.254158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] PATH_INFO: `/v3/projects/ad0d6fc02c7b4376ad7c43f2a1f710dd` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:27.255420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0d6fc02c7b4376ad7c43f2a1f710dd)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:27.257338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:27.845272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0d6fc02c7b4376ad7c43f2a1f710dd', 'request_id': 'req-d6214ac8-fc2a-41df-aee2-043172132732'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:27.855062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0d6fc02c7b4376ad7c43f2a1f710dd', 'request_id': 'req-d6214ac8-fc2a-41df-aee2-043172132732'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:28.394767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb695d-072b-4214-9715-84d96e8c03d0 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:28.395699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb695d-072b-4214-9715-84d96e8c03d0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:28.396151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb695d-072b-4214-9715-84d96e8c03d0 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:28.398130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 913/1837] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:26:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 11:26:28.405592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a77af-01c5-42ad-abe7-8eeac214a1fc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:28.405986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a77af-01c5-42ad-abe7-8eeac214a1fc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:28.406313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a77af-01c5-42ad-abe7-8eeac214a1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:28.429021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2c9a77af-01c5-42ad-abe7-8eeac214a1fc None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:28.430259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2c9a77af-01c5-42ad-abe7-8eeac214a1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:28.498799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 914/1838] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:26:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:28.511133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a77af-01c5-42ad-abe7-8eeac214a1fc None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:28.512607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2c9a77af-01c5-42ad-abe7-8eeac214a1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:28.527012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a77af-01c5-42ad-abe7-8eeac214a1fc None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:28.532752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3dd4f-7433-483b-b225-751d8b811fdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:28.533420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dd4f-7433-483b-b225-751d8b811fdc service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:28.533701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dd4f-7433-483b-b225-751d8b811fdc service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:28.533975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3dd4f-7433-483b-b225-751d8b811fdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:28.544003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3dd4f-7433-483b-b225-751d8b811fdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:28.545641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3dd4f-7433-483b-b225-751d8b811fdc service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:28.557725 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 915/1839] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:26:28.703287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0d6fc02c7b4376ad7c43f2a1f710dd', 'request_id': 'req-d6214ac8-fc2a-41df-aee2-043172132732'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:28.724548 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] The token cache is being invalidate because project ad0d6fc02c7b4376ad7c43f2a1f710dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:28.725269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:28.728230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:28.731836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 925/1840] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:27 2022] DELETE /identity/v3/projects/ad0d6fc02c7b4376ad7c43f2a1f710dd => generated 0 bytes in 1598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:28.739051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3dd4f-7433-483b-b225-751d8b811fdc service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:28.838930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3dd4f-7433-483b-b225-751d8b811fdc service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:28.845038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:28.846058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:28.846411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:28.846760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] PATH_INFO: `/v3/users/164c17d09d624a64be6211fc6b4b65e7` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:28.848639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=164c17d09d624a64be6211fc6b4b65e7)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:28.857142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:29.209075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164c17d09d624a64be6211fc6b4b65e7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:29.210522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164c17d09d624a64be6211fc6b4b65e7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:29.216405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164c17d09d624a64be6211fc6b4b65e7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:29.237752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164c17d09d624a64be6211fc6b4b65e7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:29.263783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164c17d09d624a64be6211fc6b4b65e7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:29.351232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 916/1841] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:28 2022] DELETE /identity/v3/users/164c17d09d624a64be6211fc6b4b65e7 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:29.360769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:29.853669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:29.855673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:29.875105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6214ac8-fc2a-41df-aee2-043172132732 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:29.968754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:29.969718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:29.970069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:29.970391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] PATH_INFO: `/v3/users/147fa7a0291341b081cca557740226a7` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:29.972307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=147fa7a0291341b081cca557740226a7)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:29.975690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:30.264051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '147fa7a0291341b081cca557740226a7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.265341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '147fa7a0291341b081cca557740226a7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.267105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '147fa7a0291341b081cca557740226a7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.281930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:30.284617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:30.286081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '147fa7a0291341b081cca557740226a7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.321896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-01d8c403-3912-423c-9142-8c0965d26739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '147fa7a0291341b081cca557740226a7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.326872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429c2c5-8c49-4b08-b341-c78ab03b9eef admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:30.338180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:30.344520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:30.345423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:30.345708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:30.345990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] PATH_INFO: `/v3/projects/50b0ecc0be434c5e966f00905ae89fda` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:30.350794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b0ecc0be434c5e966f00905ae89fda)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:30.353297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:30.407412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 926/1842] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:29 2022] DELETE /identity/v3/users/147fa7a0291341b081cca557740226a7 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:30.412019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e458a5-1faf-44ae-a799-074015938942 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:30.412316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e458a5-1faf-44ae-a799-074015938942 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:30.412647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e458a5-1faf-44ae-a799-074015938942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:30.466954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b5e458a5-1faf-44ae-a799-074015938942 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:30.471215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b5e458a5-1faf-44ae-a799-074015938942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:30.553755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b0ecc0be434c5e966f00905ae89fda', 'request_id': 'req-4c6b2e16-0107-4eba-8d6e-2670682215c8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.576416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b0ecc0be434c5e966f00905ae89fda', 'request_id': 'req-4c6b2e16-0107-4eba-8d6e-2670682215c8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.628137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 927/1843] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:30.646598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e458a5-1faf-44ae-a799-074015938942 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:30.649092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b5e458a5-1faf-44ae-a799-074015938942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:30.708838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b0ecc0be434c5e966f00905ae89fda', 'request_id': 'req-4c6b2e16-0107-4eba-8d6e-2670682215c8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.740612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] The token cache is being invalidate because project 50b0ecc0be434c5e966f00905ae89fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:30.741141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.742295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4c6b2e16-0107-4eba-8d6e-2670682215c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:30.745747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 917/1844] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:30 2022] DELETE /identity/v3/projects/50b0ecc0be434c5e966f00905ae89fda => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:30.750767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68dbb-a633-4917-9577-5635a9c15aaa None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:30.752729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68dbb-a633-4917-9577-5635a9c15aaa None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:30.753203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68dbb-a633-4917-9577-5635a9c15aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:30.793946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e458a5-1faf-44ae-a799-074015938942 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:30.810741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-45e68dbb-a633-4917-9577-5635a9c15aaa None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:30.814353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-45e68dbb-a633-4917-9577-5635a9c15aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:30.894930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:30.896719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:30.897210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:30.897737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:30.900957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:31.011158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:31.013363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:31.033522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 918/1845] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:31.034427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 928/1846] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:31.038702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d406f-aacc-40fa-a295-333a146dcf40 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:31.039402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d406f-aacc-40fa-a295-333a146dcf40 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:31.040276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d406f-aacc-40fa-a295-333a146dcf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:31.088065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f83d406f-aacc-40fa-a295-333a146dcf40 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:31.093139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f83d406f-aacc-40fa-a295-333a146dcf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:31.108759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 919/1847] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:31.110416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:31.124392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a93b-3022-49cf-8725-0f35b88f5cc4 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:31.138814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:31.139531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:31.139814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:31.140092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:31.140842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:31.141260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:31.142969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:31.174468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 929/1848] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:31.175977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d406f-aacc-40fa-a295-333a146dcf40 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:31.191347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d406f-aacc-40fa-a295-333a146dcf40 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:31.206122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9801ed-f2bc-4cbb-9448-0afb20a8a27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:31.206596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9801ed-f2bc-4cbb-9448-0afb20a8a27f admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:31.207868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9801ed-f2bc-4cbb-9448-0afb20a8a27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:31.208981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9801ed-f2bc-4cbb-9448-0afb20a8a27f admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:31.210240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9801ed-f2bc-4cbb-9448-0afb20a8a27f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:31.212362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9801ed-f2bc-4cbb-9448-0afb20a8a27f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:31.365531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 920/1849] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:31 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:31.369721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:31.393103 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4045-e98e-4aa8-82c9-a39512189b1f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:31.468450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2a18c-e4b3-4121-9d6c-e2f1ffb7dab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:31.469287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2a18c-e4b3-4121-9d6c-e2f1ffb7dab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:31.469782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2a18c-e4b3-4121-9d6c-e2f1ffb7dab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:31.470132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2a18c-e4b3-4121-9d6c-e2f1ffb7dab0 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:31.470847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2a18c-e4b3-4121-9d6c-e2f1ffb7dab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:31.472950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2a18c-e4b3-4121-9d6c-e2f1ffb7dab0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:31.593102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 930/1850] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:31 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:31.599317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9801ed-f2bc-4cbb-9448-0afb20a8a27f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:31.601698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:31.632862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9801ed-f2bc-4cbb-9448-0afb20a8a27f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:31.640674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c92ea-e451-4420-92a5-d3ce63a39892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:31.642112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c92ea-e451-4420-92a5-d3ce63a39892 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:31.642606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c92ea-e451-4420-92a5-d3ce63a39892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:31.643305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c92ea-e451-4420-92a5-d3ce63a39892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:31.644897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c92ea-e451-4420-92a5-d3ce63a39892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:31.647975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c92ea-e451-4420-92a5-d3ce63a39892 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:31.664179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 921/1851] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:31.668299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2a18c-e4b3-4121-9d6c-e2f1ffb7dab0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:31.680953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2a18c-e4b3-4121-9d6c-e2f1ffb7dab0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:31.696672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11f330-3c8c-4c81-b2f6-4e7597d37697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:31.697488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11f330-3c8c-4c81-b2f6-4e7597d37697 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:31.697799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11f330-3c8c-4c81-b2f6-4e7597d37697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:31.698105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11f330-3c8c-4c81-b2f6-4e7597d37697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:31.704112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11f330-3c8c-4c81-b2f6-4e7597d37697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:31.704112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11f330-3c8c-4c81-b2f6-4e7597d37697 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:31.717269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 931/1852] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:31.721481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c92ea-e451-4420-92a5-d3ce63a39892 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:31.731926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c92ea-e451-4420-92a5-d3ce63a39892 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:31.740203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:31.742031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:31.742446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:31.742805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] PATH_INFO: `/v3/projects/29093bab96c140e09d0a50389a946a7f/users/53df5c50d0404567a87f37107956d99d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:31.764941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29093bab96c140e09d0a50389a946a7f, user_id=53df5c50d0404567a87f37107956d99d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:31.768645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:31.886845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 922/1853] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:31 2022] PUT /identity/v3/projects/29093bab96c140e09d0a50389a946a7f/users/53df5c50d0404567a87f37107956d99d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:31.897531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb487c-e9de-4608-a70c-2def38c6ee68 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:31.898744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb487c-e9de-4608-a70c-2def38c6ee68 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:31.899681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb487c-e9de-4608-a70c-2def38c6ee68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:31.933016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-86fb487c-e9de-4608-a70c-2def38c6ee68 None None] MFA Rules not processed for user `53df5c50d0404567a87f37107956d99d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:31.934519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-86fb487c-e9de-4608-a70c-2def38c6ee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:31.966952 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:31.969133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:32.022760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 932/1854] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:32.023785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb487c-e9de-4608-a70c-2def38c6ee68 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:32.102444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb08683-4498-41db-b845-4b177419c5a1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:32.108334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:32.109381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.109695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.110016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] PATH_INFO: `/v3/projects/45c653038a29408e929aa46078d89493` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.112441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45c653038a29408e929aa46078d89493)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:32.114705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:32.117968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb487c-e9de-4608-a70c-2def38c6ee68 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:32.123506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb58cba-2af4-4b38-b66e-5722fd2f81c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:32.124757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb58cba-2af4-4b38-b66e-5722fd2f81c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.124757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb58cba-2af4-4b38-b66e-5722fd2f81c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.125018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb58cba-2af4-4b38-b66e-5722fd2f81c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.126299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb58cba-2af4-4b38-b66e-5722fd2f81c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:32.128361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb58cba-2af4-4b38-b66e-5722fd2f81c9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:32.251588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 933/1855] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:32 2022] POST /identity/v3/projects => generated 385 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:32.253736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb58cba-2af4-4b38-b66e-5722fd2f81c9 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:32.262797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45c653038a29408e929aa46078d89493', 'request_id': 'req-e93c8054-30c9-4c76-a23f-8496df4cc7c0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.270185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45c653038a29408e929aa46078d89493', 'request_id': 'req-e93c8054-30c9-4c76-a23f-8496df4cc7c0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.316952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb58cba-2af4-4b38-b66e-5722fd2f81c9 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:32.321657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:32.322332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.322620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.322897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.323623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:32.325540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:32.382716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45c653038a29408e929aa46078d89493', 'request_id': 'req-e93c8054-30c9-4c76-a23f-8496df4cc7c0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.395889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] The token cache is being invalidate because project 45c653038a29408e929aa46078d89493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:32.396210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.396890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.398629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 923/1856] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:31 2022] DELETE /identity/v3/projects/45c653038a29408e929aa46078d89493 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:32.402517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:32.403888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:32.465367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 934/1857] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:32 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:32.474545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:32.474545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:32.492929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303be87e-942e-4934-9311-0d7c28d54b15 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:32.508819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac9092-be01-49af-b9b7-c956682e0be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:32.508819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac9092-be01-49af-b9b7-c956682e0be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.508819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac9092-be01-49af-b9b7-c956682e0be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.508819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac9092-be01-49af-b9b7-c956682e0be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.509309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac9092-be01-49af-b9b7-c956682e0be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:32.510022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac9092-be01-49af-b9b7-c956682e0be4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:32.510660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c8054-30c9-4c76-a23f-8496df4cc7c0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:32.518429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:32.520744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.521034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.521311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] PATH_INFO: `/v3/users/f69da77f50c940a482f5d43d9341e8c5` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.523806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 935/1858] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:32.524132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69da77f50c940a482f5d43d9341e8c5)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:32.526321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:32.527967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac9092-be01-49af-b9b7-c956682e0be4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:32.538722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac9092-be01-49af-b9b7-c956682e0be4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:32.543400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37435217-9ce3-4efd-aa0f-3bc58242db3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:32.544303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37435217-9ce3-4efd-aa0f-3bc58242db3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.556750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37435217-9ce3-4efd-aa0f-3bc58242db3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.556750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37435217-9ce3-4efd-aa0f-3bc58242db3f admin admin] PATH_INFO: `/v3/projects/87b6221d25ec49bfbae07cf886fe0886/users/dad3db0c6cd242d19a86dfd0e10b7dbc/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.565855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37435217-9ce3-4efd-aa0f-3bc58242db3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b6221d25ec49bfbae07cf886fe0886, user_id=dad3db0c6cd242d19a86dfd0e10b7dbc, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:32.568665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37435217-9ce3-4efd-aa0f-3bc58242db3f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:32.687677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 936/1859] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:32 2022] PUT /identity/v3/projects/87b6221d25ec49bfbae07cf886fe0886/users/dad3db0c6cd242d19a86dfd0e10b7dbc/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:32.714381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37435217-9ce3-4efd-aa0f-3bc58242db3f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:32.755848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69da77f50c940a482f5d43d9341e8c5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.757366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69da77f50c940a482f5d43d9341e8c5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.758161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69da77f50c940a482f5d43d9341e8c5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.768393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:32.768658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69da77f50c940a482f5d43d9341e8c5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.783426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69da77f50c940a482f5d43d9341e8c5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:32.790302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37435217-9ce3-4efd-aa0f-3bc58242db3f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:32.805375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:32.806155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.806519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.806859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.807722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:32.809498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:32.824514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 937/1860] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:26:32.832466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:32.834145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:32.848481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3e44f-ec0d-4f5c-8448-ed778a27d812 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:32.856840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998e76b-cac4-49fa-a954-afc801b9fc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:32.858473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998e76b-cac4-49fa-a954-afc801b9fc2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.858473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998e76b-cac4-49fa-a954-afc801b9fc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.858693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998e76b-cac4-49fa-a954-afc801b9fc2f admin admin] PATH_INFO: `/v3/projects/87b6221d25ec49bfbae07cf886fe0886/users/dad3db0c6cd242d19a86dfd0e10b7dbc/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.860748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998e76b-cac4-49fa-a954-afc801b9fc2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b6221d25ec49bfbae07cf886fe0886, user_id=dad3db0c6cd242d19a86dfd0e10b7dbc, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:32.863572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998e76b-cac4-49fa-a954-afc801b9fc2f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:32.876229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 938/1861] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:32 2022] PUT /identity/v3/projects/87b6221d25ec49bfbae07cf886fe0886/users/dad3db0c6cd242d19a86dfd0e10b7dbc/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:26:32.890870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 924/1862] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:32 2022] DELETE /identity/v3/users/f69da77f50c940a482f5d43d9341e8c5 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:32.899122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c6ff-46cb-4ff8-8d6c-572913ddda60 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:32.899665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c6ff-46cb-4ff8-8d6c-572913ddda60 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:32.899748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6c6ff-46cb-4ff8-8d6c-572913ddda60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:32.930380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6df6c6ff-46cb-4ff8-8d6c-572913ddda60 None None] MFA Rules not processed for user `dad3db0c6cd242d19a86dfd0e10b7dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:32.931682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6df6c6ff-46cb-4ff8-8d6c-572913ddda60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:33.008974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 939/1863] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:33.021020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:33.026898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:33.142925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4467332-63ba-46c5-be8d-6407b03d7da4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:33.156646 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855147c-70b6-40c2-89cc-0ce9bb937373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:33.157338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855147c-70b6-40c2-89cc-0ce9bb937373 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:33.158427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855147c-70b6-40c2-89cc-0ce9bb937373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:33.158734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855147c-70b6-40c2-89cc-0ce9bb937373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:33.174093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855147c-70b6-40c2-89cc-0ce9bb937373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:33.176743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855147c-70b6-40c2-89cc-0ce9bb937373 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:33.189708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 925/1864] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:34.081787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6c6ff-46cb-4ff8-8d6c-572913ddda60 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:34.083420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6df6c6ff-46cb-4ff8-8d6c-572913ddda60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:34.156348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6c6ff-46cb-4ff8-8d6c-572913ddda60 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:34.161437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:34.162317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:34.162591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:34.162869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] PATH_INFO: `/v3/projects/7f088e9b5cf048b08136abf8574d804b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:34.164090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f088e9b5cf048b08136abf8574d804b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:34.166036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:34.286124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:34.307461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f088e9b5cf048b08136abf8574d804b', 'request_id': 'req-434631fc-f5b8-4108-b31e-93b48428072b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:34.313269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f088e9b5cf048b08136abf8574d804b', 'request_id': 'req-434631fc-f5b8-4108-b31e-93b48428072b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:34.376865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f088e9b5cf048b08136abf8574d804b', 'request_id': 'req-434631fc-f5b8-4108-b31e-93b48428072b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:34.388134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] The token cache is being invalidate because project 7f088e9b5cf048b08136abf8574d804b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:34.388426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:34.389121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-434631fc-f5b8-4108-b31e-93b48428072b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:34.390683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 940/1865] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:34 2022] DELETE /identity/v3/projects/7f088e9b5cf048b08136abf8574d804b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:34.404870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab0a96-d70c-443f-bba0-50bd3215c3ec None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:34.404870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab0a96-d70c-443f-bba0-50bd3215c3ec None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:34.404870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab0a96-d70c-443f-bba0-50bd3215c3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:34.423005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4aab0a96-d70c-443f-bba0-50bd3215c3ec None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:34.424264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4aab0a96-d70c-443f-bba0-50bd3215c3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:34.491483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 926/1866] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:34.499755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29127f1-82d2-45ba-a84f-b1692cdb3a47 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:34.500054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29127f1-82d2-45ba-a84f-b1692cdb3a47 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:34.500392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29127f1-82d2-45ba-a84f-b1692cdb3a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:34.518808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a29127f1-82d2-45ba-a84f-b1692cdb3a47 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:34.520069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a29127f1-82d2-45ba-a84f-b1692cdb3a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:34.528569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 941/1867] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:34.539039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab0a96-d70c-443f-bba0-50bd3215c3ec None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:34.549341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab0a96-d70c-443f-bba0-50bd3215c3ec None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:34.554235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:34.554895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:34.555311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:34.555592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:34.556309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:34.556619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:34.558210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:34.570729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 927/1868] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:34.590596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29127f1-82d2-45ba-a84f-b1692cdb3a47 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:34.599951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29127f1-82d2-45ba-a84f-b1692cdb3a47 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:34.607791 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:34.608462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:34.608747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:34.609024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:34.610258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:34.615973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:34.746800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 942/1869] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:34 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:34.763339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:34.768804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88d275-9763-4379-b4eb-07508fc20098 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:34.795043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:34.796852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:34.865245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:34.866894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:34.867239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:34.867660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:34.868319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:34.869642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ec75d-a711-4f5c-996b-a8676dc8040d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:34.870456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:34.875088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:34.876427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:34.877049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:34.878474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:34.881814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:34.962947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 928/1870] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:34 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:34.963758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:34.970874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:34.999696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2ac93-69ee-48b2-84e5-ed75fa2a58b2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:35.004903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:35.007938 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:35.010180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:35.019328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.019328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.019328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] PATH_INFO: `/v3/users/53df5c50d0404567a87f37107956d99d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.021635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53df5c50d0404567a87f37107956d99d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:35.024484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:35.050367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 943/1871] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:26:35.051864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:35.089241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8e978-a627-4eed-a605-6f45e6347ed1 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:35.097307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24005c6-52c8-414b-a6fb-18cf088bf7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:35.098334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24005c6-52c8-414b-a6fb-18cf088bf7db admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.098850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24005c6-52c8-414b-a6fb-18cf088bf7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.099496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24005c6-52c8-414b-a6fb-18cf088bf7db admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.101892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24005c6-52c8-414b-a6fb-18cf088bf7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:35.104522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24005c6-52c8-414b-a6fb-18cf088bf7db admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:35.121118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 944/1872] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:35.126201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24005c6-52c8-414b-a6fb-18cf088bf7db admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:35.144317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24005c6-52c8-414b-a6fb-18cf088bf7db admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:35.157199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d06bf-bb6c-412a-bdcb-72a20b7a2149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:35.157905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d06bf-bb6c-412a-bdcb-72a20b7a2149 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.158193 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d06bf-bb6c-412a-bdcb-72a20b7a2149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.158466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d06bf-bb6c-412a-bdcb-72a20b7a2149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.159291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d06bf-bb6c-412a-bdcb-72a20b7a2149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:35.161045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d06bf-bb6c-412a-bdcb-72a20b7a2149 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:35.184951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 945/1873] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:35.186695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d06bf-bb6c-412a-bdcb-72a20b7a2149 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:35.203515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d06bf-bb6c-412a-bdcb-72a20b7a2149 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:35.210008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acb4d6-438c-477b-a872-cac801bb467a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:35.211007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acb4d6-438c-477b-a872-cac801bb467a admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.211311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acb4d6-438c-477b-a872-cac801bb467a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.211594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acb4d6-438c-477b-a872-cac801bb467a admin admin] PATH_INFO: `/v3/projects/99fbbf8086054b439e40a3d146b9b4bc/users/b46bc44f354a4e4380c225953fad2c21/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.260800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acb4d6-438c-477b-a872-cac801bb467a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fbbf8086054b439e40a3d146b9b4bc, user_id=b46bc44f354a4e4380c225953fad2c21, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:35.264128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acb4d6-438c-477b-a872-cac801bb467a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:35.345655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53df5c50d0404567a87f37107956d99d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:35.346750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53df5c50d0404567a87f37107956d99d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:35.347535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53df5c50d0404567a87f37107956d99d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:35.354568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53df5c50d0404567a87f37107956d99d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:35.368838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-455fda8e-fab7-47de-9d75-bfa6d06625c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53df5c50d0404567a87f37107956d99d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:35.406504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 946/1874] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:35 2022] PUT /identity/v3/projects/99fbbf8086054b439e40a3d146b9b4bc/users/b46bc44f354a4e4380c225953fad2c21/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:35.408295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b4c2-26e6-40ab-b93f-42b880d17bb9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.408592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b4c2-26e6-40ab-b93f-42b880d17bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.408879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118b4c2-26e6-40ab-b93f-42b880d17bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.439436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4118b4c2-26e6-40ab-b93f-42b880d17bb9 None None] MFA Rules not processed for user `b46bc44f354a4e4380c225953fad2c21`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:35.440847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4118b4c2-26e6-40ab-b93f-42b880d17bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:35.503659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 929/1875] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:34 2022] DELETE /identity/v3/users/53df5c50d0404567a87f37107956d99d => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:35.508163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4027-3718-434e-bbc9-b9f59a6eb11d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.509186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4027-3718-434e-bbc9-b9f59a6eb11d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.509186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4027-3718-434e-bbc9-b9f59a6eb11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.512949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 947/1876] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:35.524382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b4c2-26e6-40ab-b93f-42b880d17bb9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:35.525824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4118b4c2-26e6-40ab-b93f-42b880d17bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:35.530837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-24ff4027-3718-434e-bbc9-b9f59a6eb11d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:35.532539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-24ff4027-3718-434e-bbc9-b9f59a6eb11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:35.591217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118b4c2-26e6-40ab-b93f-42b880d17bb9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:35.592313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 930/1877] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:35.602629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4027-3718-434e-bbc9-b9f59a6eb11d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:35.604123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-24ff4027-3718-434e-bbc9-b9f59a6eb11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:35.641096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79910acc-e8d3-4e75-a22f-8c76cb33922a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:35.642864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79910acc-e8d3-4e75-a22f-8c76cb33922a service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.643171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79910acc-e8d3-4e75-a22f-8c76cb33922a service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.643481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79910acc-e8d3-4e75-a22f-8c76cb33922a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.663593 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4027-3718-434e-bbc9-b9f59a6eb11d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:35.667271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:35.667947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.668204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.668480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.678544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:35.680905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:35.691316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 931/1878] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:35.708837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79910acc-e8d3-4e75-a22f-8c76cb33922a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:35.711098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79910acc-e8d3-4e75-a22f-8c76cb33922a service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:35.725298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 948/1879] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:26:35.748524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:35.750581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:35.766881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491814a-5ee5-42e3-b6ed-127568985a07 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:35.772696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:35.774076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:35.774578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:35.774872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] PATH_INFO: `/v3/users/b46bc44f354a4e4380c225953fad2c21` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:35.778294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b46bc44f354a4e4380c225953fad2c21)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:35.780390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:36.020080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b46bc44f354a4e4380c225953fad2c21'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.020681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b46bc44f354a4e4380c225953fad2c21'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.021303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b46bc44f354a4e4380c225953fad2c21'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.029228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b46bc44f354a4e4380c225953fad2c21'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.042353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b46bc44f354a4e4380c225953fad2c21'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.150883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 932/1880] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:35 2022] DELETE /identity/v3/users/b46bc44f354a4e4380c225953fad2c21 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:36.156131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1851a8-aa36-4a13-9df7-468947855876 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:36.157186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1851a8-aa36-4a13-9df7-468947855876 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:36.157660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1851a8-aa36-4a13-9df7-468947855876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:36.178175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fb1851a8-aa36-4a13-9df7-468947855876 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:36.179606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fb1851a8-aa36-4a13-9df7-468947855876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:36.276352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 949/1881] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:36.288769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:36.290714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:36.356723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6a1f21-5724-4b8c-afc5-dd3edbcfb53f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:36.361514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2aa1-b525-497c-92af-027c81b6964b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:36.362240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2aa1-b525-497c-92af-027c81b6964b service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:36.362528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2aa1-b525-497c-92af-027c81b6964b service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:36.362804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e2aa1-b525-497c-92af-027c81b6964b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:36.372728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e2aa1-b525-497c-92af-027c81b6964b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:36.375098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e2aa1-b525-497c-92af-027c81b6964b service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:36.385397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 933/1882] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:26:36.443179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1851a8-aa36-4a13-9df7-468947855876 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:36.445871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fb1851a8-aa36-4a13-9df7-468947855876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:36.473620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1851a8-aa36-4a13-9df7-468947855876 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:36.482192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:36.483485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:36.484009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:36.484784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] PATH_INFO: `/v3/projects/29093bab96c140e09d0a50389a946a7f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:36.486251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29093bab96c140e09d0a50389a946a7f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:36.502949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:36.680117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29093bab96c140e09d0a50389a946a7f', 'request_id': 'req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.694698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29093bab96c140e09d0a50389a946a7f', 'request_id': 'req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.818672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29093bab96c140e09d0a50389a946a7f', 'request_id': 'req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.836437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] The token cache is being invalidate because project 29093bab96c140e09d0a50389a946a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:36.837101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.838042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:36.840377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 950/1883] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:36 2022] DELETE /identity/v3/projects/29093bab96c140e09d0a50389a946a7f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:36.847928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2aa1-b525-497c-92af-027c81b6964b service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:36.862438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e2aa1-b525-497c-92af-027c81b6964b service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:36.948415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:36.950875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:36.951398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:36.952118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] PATH_INFO: `/v3/users/dad3db0c6cd242d19a86dfd0e10b7dbc` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:36.956125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad3db0c6cd242d19a86dfd0e10b7dbc)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:36.957732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:36.964011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:36.964455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:36.992564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b6e6f-b3fe-489d-97dc-525a9b2b7f72 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:37.000708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:37.001683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:37.002070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:37.002435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] PATH_INFO: `/v3/projects/99fbbf8086054b439e40a3d146b9b4bc` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:37.003747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fbbf8086054b439e40a3d146b9b4bc)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:37.006322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:37.224591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:37.243079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fbbf8086054b439e40a3d146b9b4bc', 'request_id': 'req-25d0d2db-cda4-4884-8650-26ff5aaeef15'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.244426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad3db0c6cd242d19a86dfd0e10b7dbc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.260943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad3db0c6cd242d19a86dfd0e10b7dbc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.262332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad3db0c6cd242d19a86dfd0e10b7dbc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.263100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fbbf8086054b439e40a3d146b9b4bc', 'request_id': 'req-25d0d2db-cda4-4884-8650-26ff5aaeef15'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.276287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad3db0c6cd242d19a86dfd0e10b7dbc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.291681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad3db0c6cd242d19a86dfd0e10b7dbc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.428415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fbbf8086054b439e40a3d146b9b4bc', 'request_id': 'req-25d0d2db-cda4-4884-8650-26ff5aaeef15'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.450590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] The token cache is being invalidate because project 99fbbf8086054b439e40a3d146b9b4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:37.450590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.465479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-25d0d2db-cda4-4884-8650-26ff5aaeef15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:37.465479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 951/1884] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:36 2022] DELETE /identity/v3/projects/99fbbf8086054b439e40a3d146b9b4bc => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:37.468853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698511e9-c4a7-41ca-832e-25b0adc5abef None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:37.469147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698511e9-c4a7-41ca-832e-25b0adc5abef None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:37.469424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698511e9-c4a7-41ca-832e-25b0adc5abef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:37.488340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-698511e9-c4a7-41ca-832e-25b0adc5abef None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:37.489560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-698511e9-c4a7-41ca-832e-25b0adc5abef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:37.553173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 952/1885] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:37.562155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a645-6244-4396-8dfd-4d3c53fdddd0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:37.562453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a645-6244-4396-8dfd-4d3c53fdddd0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:37.563660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1a645-6244-4396-8dfd-4d3c53fdddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:37.575417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:37.589041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 934/1886] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:36 2022] DELETE /identity/v3/users/dad3db0c6cd242d19a86dfd0e10b7dbc => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:37.601686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-39a1a645-6244-4396-8dfd-4d3c53fdddd0 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:37.620997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-39a1a645-6244-4396-8dfd-4d3c53fdddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:37.678626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 953/1887] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:37.682675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:37.684487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:37.698314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f678e18b-ecb9-4f85-8133-43276b278f79 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:37.702902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:37.703711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:37.703998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:37.704273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:37.705004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:37.705310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:37.706926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:37.718971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 935/1888] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:26:37.723058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a645-6244-4396-8dfd-4d3c53fdddd0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:37.733622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1a645-6244-4396-8dfd-4d3c53fdddd0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:37.739954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edb59c-1a20-48c2-a178-e72b111f9d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:37.740631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb59c-1a20-48c2-a178-e72b111f9d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:37.740982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb59c-1a20-48c2-a178-e72b111f9d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:37.741263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edb59c-1a20-48c2-a178-e72b111f9d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:37.741997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edb59c-1a20-48c2-a178-e72b111f9d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:37.744289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79edb59c-1a20-48c2-a178-e72b111f9d24 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:37.905177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 954/1889] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:37 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:37.912084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:37.923352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb02c75-f043-4443-bfc7-a0046a1ad1f0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:37.990194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16b10e-1198-4e87-8149-8866e58902e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:37.996154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16b10e-1198-4e87-8149-8866e58902e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:37.996154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16b10e-1198-4e87-8149-8866e58902e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:37.996154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16b10e-1198-4e87-8149-8866e58902e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:37.996154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16b10e-1198-4e87-8149-8866e58902e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:37.996154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16b10e-1198-4e87-8149-8866e58902e6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:38.132089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 936/1890] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:37 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:38.136409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edb59c-1a20-48c2-a178-e72b111f9d24 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:38.147169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edb59c-1a20-48c2-a178-e72b111f9d24 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:38.151631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033211c1-c848-4fc5-9971-f8ba79bf923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:38.152291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033211c1-c848-4fc5-9971-f8ba79bf923a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.152574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033211c1-c848-4fc5-9971-f8ba79bf923a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.152879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033211c1-c848-4fc5-9971-f8ba79bf923a admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.153596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033211c1-c848-4fc5-9971-f8ba79bf923a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:38.155174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033211c1-c848-4fc5-9971-f8ba79bf923a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:38.165478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 955/1891] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:26:38.169770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16b10e-1198-4e87-8149-8866e58902e6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:38.180362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16b10e-1198-4e87-8149-8866e58902e6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:38.184969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:38.185628 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.185906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.186526 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.186903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:38.189113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:38.202277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 937/1892] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:38.203377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033211c1-c848-4fc5-9971-f8ba79bf923a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:38.213266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033211c1-c848-4fc5-9971-f8ba79bf923a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:38.218205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa1c08-0d05-475e-ad90-3e27cd7bc3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:38.219363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa1c08-0d05-475e-ad90-3e27cd7bc3d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.219648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa1c08-0d05-475e-ad90-3e27cd7bc3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.219923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa1c08-0d05-475e-ad90-3e27cd7bc3d7 admin admin] PATH_INFO: `/v3/projects/ebc6d477259b4c4880e64819918ffbbc/users/5504e395d523412c9626e57a99939f3d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.237653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:38.239794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:38.247476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa1c08-0d05-475e-ad90-3e27cd7bc3d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc6d477259b4c4880e64819918ffbbc, user_id=5504e395d523412c9626e57a99939f3d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:38.251097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa1c08-0d05-475e-ad90-3e27cd7bc3d7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:38.255186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63422766-9f67-4471-8dd7-82e96925aaa5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:38.260072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:38.262782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.262782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.262782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] PATH_INFO: `/v3/projects/87b6221d25ec49bfbae07cf886fe0886` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.262782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b6221d25ec49bfbae07cf886fe0886)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:38.264612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:38.343420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 956/1893] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:38 2022] PUT /identity/v3/projects/ebc6d477259b4c4880e64819918ffbbc/users/5504e395d523412c9626e57a99939f3d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:38.353371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783a9d8-d150-41fe-9a71-62a10688b493 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.353664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783a9d8-d150-41fe-9a71-62a10688b493 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.353945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783a9d8-d150-41fe-9a71-62a10688b493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.389966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2783a9d8-d150-41fe-9a71-62a10688b493 None None] MFA Rules not processed for user `5504e395d523412c9626e57a99939f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:38.391408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2783a9d8-d150-41fe-9a71-62a10688b493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:38.424302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b6221d25ec49bfbae07cf886fe0886', 'request_id': 'req-df18440b-20e9-4286-9d9c-173b82498a6a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:38.430629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b6221d25ec49bfbae07cf886fe0886', 'request_id': 'req-df18440b-20e9-4286-9d9c-173b82498a6a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:38.486116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 957/1894] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:38.497048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783a9d8-d150-41fe-9a71-62a10688b493 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:38.500937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2783a9d8-d150-41fe-9a71-62a10688b493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:38.514405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b6221d25ec49bfbae07cf886fe0886', 'request_id': 'req-df18440b-20e9-4286-9d9c-173b82498a6a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:38.538046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] The token cache is being invalidate because project 87b6221d25ec49bfbae07cf886fe0886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:38.538648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:38.539620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-df18440b-20e9-4286-9d9c-173b82498a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:38.545475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 938/1895] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:38 2022] DELETE /identity/v3/projects/87b6221d25ec49bfbae07cf886fe0886 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:26:38.547820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620165-079c-4b91-a525-fd6bc4a72fc7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.548210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620165-079c-4b91-a525-fd6bc4a72fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.548564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5620165-079c-4b91-a525-fd6bc4a72fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.567926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f5620165-079c-4b91-a525-fd6bc4a72fc7 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:38.569287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f5620165-079c-4b91-a525-fd6bc4a72fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:38.594783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783a9d8-d150-41fe-9a71-62a10688b493 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:38.651881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 939/1896] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:38.654405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da2a7-b2c8-4864-83a2-94044e373a38 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.656756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da2a7-b2c8-4864-83a2-94044e373a38 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.657938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856da2a7-b2c8-4864-83a2-94044e373a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.670525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:38.671211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.671542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.671854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.679201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:38.686616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-856da2a7-b2c8-4864-83a2-94044e373a38 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:38.688979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-856da2a7-b2c8-4864-83a2-94044e373a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:38.704701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 940/1897] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:38.706074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856da2a7-b2c8-4864-83a2-94044e373a38 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:38.719847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856da2a7-b2c8-4864-83a2-94044e373a38 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:38.728644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:38.731527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.731948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.733118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.734017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:38.735278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:38.738694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:38.759109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 941/1898] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:38.764591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:38.769607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:38.771088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:38.779906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37ff22-0e81-487b-aeff-f95e8b85c7ba admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:38.798230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bf439-f881-475b-99d1-3632013f81a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:38.799425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bf439-f881-475b-99d1-3632013f81a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:38.799802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bf439-f881-475b-99d1-3632013f81a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:38.800236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bf439-f881-475b-99d1-3632013f81a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:38.801945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bf439-f881-475b-99d1-3632013f81a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:38.808148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bf439-f881-475b-99d1-3632013f81a9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:38.808713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 958/1899] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:38.915479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 942/1900] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:38 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:38.920786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:38.935675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cc44e-ab91-409a-ba7c-812b5fcd3407 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:39.030453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744f9fb-dc86-4fc3-afde-40daa85a69a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:39.031866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f9fb-dc86-4fc3-afde-40daa85a69a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:39.032694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f9fb-dc86-4fc3-afde-40daa85a69a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:39.032694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f9fb-dc86-4fc3-afde-40daa85a69a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:39.033187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744f9fb-dc86-4fc3-afde-40daa85a69a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:39.035173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744f9fb-dc86-4fc3-afde-40daa85a69a7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:39.220123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 959/1901] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:38 2022] POST /identity/v3/users => generated 292 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:39.220225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bf439-f881-475b-99d1-3632013f81a9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:39.230887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bf439-f881-475b-99d1-3632013f81a9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:39.237967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f2fca-28d7-4eea-bf23-10015738c23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:39.238740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2fca-28d7-4eea-bf23-10015738c23d admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:39.239021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2fca-28d7-4eea-bf23-10015738c23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:39.239348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f2fca-28d7-4eea-bf23-10015738c23d admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:39.240442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f2fca-28d7-4eea-bf23-10015738c23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:39.241740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f2fca-28d7-4eea-bf23-10015738c23d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:39.252425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 943/1902] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:39.256425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744f9fb-dc86-4fc3-afde-40daa85a69a7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:39.266604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744f9fb-dc86-4fc3-afde-40daa85a69a7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:39.271407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372252fc-8c4a-481b-9503-146082c80235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:39.272112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372252fc-8c4a-481b-9503-146082c80235 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:39.272396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372252fc-8c4a-481b-9503-146082c80235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:39.272668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372252fc-8c4a-481b-9503-146082c80235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:39.273422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372252fc-8c4a-481b-9503-146082c80235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:39.275016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372252fc-8c4a-481b-9503-146082c80235 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:39.297850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 960/1903] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:39.307717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f2fca-28d7-4eea-bf23-10015738c23d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:39.318561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f2fca-28d7-4eea-bf23-10015738c23d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:39.323672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:39.325095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:39.325395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:39.325680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] PATH_INFO: `/v3/projects/769c162981dd42319f0e0d79f2b7fd34/users/bfbe689c9b414eaead0ab23efb60bf11/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:39.341570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769c162981dd42319f0e0d79f2b7fd34, user_id=bfbe689c9b414eaead0ab23efb60bf11, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:39.345415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:39.426516 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 944/1904] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:39 2022] PUT /identity/v3/projects/769c162981dd42319f0e0d79f2b7fd34/users/bfbe689c9b414eaead0ab23efb60bf11/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:39.436601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e3d2f-b495-49b1-b36f-a132e6b01adb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:39.436888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e3d2f-b495-49b1-b36f-a132e6b01adb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:39.437206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e3d2f-b495-49b1-b36f-a132e6b01adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:39.468551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5d4e3d2f-b495-49b1-b36f-a132e6b01adb None None] MFA Rules not processed for user `bfbe689c9b414eaead0ab23efb60bf11`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:39.469832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5d4e3d2f-b495-49b1-b36f-a132e6b01adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:39.558528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 961/1905] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:39.571442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:39.573127 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:39.633520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55ff95-772a-433b-97d2-646b9a1cef11 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:39.637203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e13e0-9447-4bbf-8fc1-29c926e5bee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:39.637861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e13e0-9447-4bbf-8fc1-29c926e5bee7 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:39.638146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e13e0-9447-4bbf-8fc1-29c926e5bee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:39.638427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e13e0-9447-4bbf-8fc1-29c926e5bee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:39.656772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e13e0-9447-4bbf-8fc1-29c926e5bee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:39.656772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e13e0-9447-4bbf-8fc1-29c926e5bee7 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:39.673198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 945/1906] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:39.682340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e3d2f-b495-49b1-b36f-a132e6b01adb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:39.758204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e3d2f-b495-49b1-b36f-a132e6b01adb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:39.763475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:39.764494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:39.764847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:39.765194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] PATH_INFO: `/v3/users/bfbe689c9b414eaead0ab23efb60bf11` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:39.766617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfbe689c9b414eaead0ab23efb60bf11)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:39.768826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:40.093245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfbe689c9b414eaead0ab23efb60bf11'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:40.094463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfbe689c9b414eaead0ab23efb60bf11'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:40.095435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfbe689c9b414eaead0ab23efb60bf11'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:40.113449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfbe689c9b414eaead0ab23efb60bf11'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:40.126682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfbe689c9b414eaead0ab23efb60bf11'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:40.230498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 962/1907] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:39 2022] DELETE /identity/v3/users/bfbe689c9b414eaead0ab23efb60bf11 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:40.231989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d0abc-fb36-4dba-af1c-dcd753403c57 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:40.232425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d0abc-fb36-4dba-af1c-dcd753403c57 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:40.232817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d0abc-fb36-4dba-af1c-dcd753403c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:40.262121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cb3d0abc-fb36-4dba-af1c-dcd753403c57 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:40.263674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cb3d0abc-fb36-4dba-af1c-dcd753403c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:40.361898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 946/1908] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:40.408881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:40.411020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:40.476763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697a2ca-69e3-44fc-9279-06ce284383a4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:40.481088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ae612-41dc-49c1-9a53-8deca8867c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:40.481747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ae612-41dc-49c1-9a53-8deca8867c30 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:40.482031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ae612-41dc-49c1-9a53-8deca8867c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:40.482308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ae612-41dc-49c1-9a53-8deca8867c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:40.493278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ae612-41dc-49c1-9a53-8deca8867c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:40.494778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ae612-41dc-49c1-9a53-8deca8867c30 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:40.506010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 963/1909] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:40.852955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:41.166012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:41.413344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d0abc-fb36-4dba-af1c-dcd753403c57 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:41.414770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cb3d0abc-fb36-4dba-af1c-dcd753403c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:41.427820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d0abc-fb36-4dba-af1c-dcd753403c57 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:41.433935 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:41.435027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:41.435539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:41.436116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] PATH_INFO: `/v3/projects/769c162981dd42319f0e0d79f2b7fd34` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:41.437595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=769c162981dd42319f0e0d79f2b7fd34)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:41.439714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:41.589883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '769c162981dd42319f0e0d79f2b7fd34', 'request_id': 'req-dce67205-3ae4-4d3b-b47d-62eb73681323'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:41.597014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '769c162981dd42319f0e0d79f2b7fd34', 'request_id': 'req-dce67205-3ae4-4d3b-b47d-62eb73681323'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:41.696495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '769c162981dd42319f0e0d79f2b7fd34', 'request_id': 'req-dce67205-3ae4-4d3b-b47d-62eb73681323'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:41.710974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] The token cache is being invalidate because project 769c162981dd42319f0e0d79f2b7fd34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:41.711621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:41.712509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-dce67205-3ae4-4d3b-b47d-62eb73681323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:41.714343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 947/1910] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:41 2022] DELETE /identity/v3/projects/769c162981dd42319f0e0d79f2b7fd34 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:41.720063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce093-f8d6-4fa2-b8fd-509f0a8cb344 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:41.720545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce093-f8d6-4fa2-b8fd-509f0a8cb344 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:41.720634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce093-f8d6-4fa2-b8fd-509f0a8cb344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:41.743280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-156ce093-f8d6-4fa2-b8fd-509f0a8cb344 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:41.744653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-156ce093-f8d6-4fa2-b8fd-509f0a8cb344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:41.812936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 964/1911] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:41.822190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed70c4-5fc4-4167-9ea1-d05ed17bb0bb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:41.824210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed70c4-5fc4-4167-9ea1-d05ed17bb0bb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:41.824210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed70c4-5fc4-4167-9ea1-d05ed17bb0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:41.837045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce093-f8d6-4fa2-b8fd-509f0a8cb344 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:41.843327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b4ed70c4-5fc4-4167-9ea1-d05ed17bb0bb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:41.845090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b4ed70c4-5fc4-4167-9ea1-d05ed17bb0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:41.851217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-156ce093-f8d6-4fa2-b8fd-509f0a8cb344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:41.860314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 948/1912] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:41.865593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce093-f8d6-4fa2-b8fd-509f0a8cb344 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:41.867537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed70c4-5fc4-4167-9ea1-d05ed17bb0bb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:41.872417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:41.873311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:41.873592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:41.874053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] PATH_INFO: `/v3/users/134a94416f12451eb07942b8a797c35d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:41.875364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=134a94416f12451eb07942b8a797c35d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:41.877508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:41.879099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed70c4-5fc4-4167-9ea1-d05ed17bb0bb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:41.888800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:41.889477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:41.889760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:41.890090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:41.891491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:41.891971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:41.893576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:41.912855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 949/1913] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:41.918311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:41.929648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c979e-f459-485e-bb6c-bbbf6f8960e0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:41.934383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea8ecd-1b16-4838-b21f-09403805a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:41.935120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea8ecd-1b16-4838-b21f-09403805a4dc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:41.935509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea8ecd-1b16-4838-b21f-09403805a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:41.935786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea8ecd-1b16-4838-b21f-09403805a4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:41.936794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea8ecd-1b16-4838-b21f-09403805a4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:41.938839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea8ecd-1b16-4838-b21f-09403805a4dc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:42.079807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 950/1914] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:41 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:42.084208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea8ecd-1b16-4838-b21f-09403805a4dc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:42.130637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134a94416f12451eb07942b8a797c35d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:42.131393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134a94416f12451eb07942b8a797c35d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:42.131983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134a94416f12451eb07942b8a797c35d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:42.142776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134a94416f12451eb07942b8a797c35d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:42.155465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ca83ee31-445e-4b73-ad89-2fdad9594b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134a94416f12451eb07942b8a797c35d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:42.157827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea8ecd-1b16-4838-b21f-09403805a4dc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:42.162883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:42.164068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:42.164510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:42.164935 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:42.165830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:42.168305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:42.234827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 965/1915] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:41 2022] DELETE /identity/v3/users/134a94416f12451eb07942b8a797c35d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:42.239536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9b59e-a828-4afd-9850-a8c6835398c8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:42.239849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9b59e-a828-4afd-9850-a8c6835398c8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:42.240136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9b59e-a828-4afd-9850-a8c6835398c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:42.263816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-01a9b59e-a828-4afd-9850-a8c6835398c8 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:42.264782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-01a9b59e-a828-4afd-9850-a8c6835398c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:42.269329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 951/1916] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:42 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:42.273525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:42.275238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:42.294146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096f837-085a-449f-82b0-917b653dcefa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:42.341529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 966/1917] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:42.352001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9b59e-a828-4afd-9850-a8c6835398c8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:42.353394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-01a9b59e-a828-4afd-9850-a8c6835398c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:42.372637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f060129-23a3-44dd-9334-ce5afde69b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:42.373463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f060129-23a3-44dd-9334-ce5afde69b27 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:42.374014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f060129-23a3-44dd-9334-ce5afde69b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:42.374390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f060129-23a3-44dd-9334-ce5afde69b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:42.378274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f060129-23a3-44dd-9334-ce5afde69b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:42.378274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f060129-23a3-44dd-9334-ce5afde69b27 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:42.401970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 952/1918] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:26:42.409045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f060129-23a3-44dd-9334-ce5afde69b27 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:42.423317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f060129-23a3-44dd-9334-ce5afde69b27 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:42.432378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757583f-1b39-4868-b2a8-d0d0bacfc37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:42.433433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757583f-1b39-4868-b2a8-d0d0bacfc37a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:42.433795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757583f-1b39-4868-b2a8-d0d0bacfc37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:42.434194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757583f-1b39-4868-b2a8-d0d0bacfc37a admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:42.435658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757583f-1b39-4868-b2a8-d0d0bacfc37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:42.437763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757583f-1b39-4868-b2a8-d0d0bacfc37a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:42.442725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9b59e-a828-4afd-9850-a8c6835398c8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:42.447577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a527f8c-0ad2-4fde-9893-7a41b40d09b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:42.448438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527f8c-0ad2-4fde-9893-7a41b40d09b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:42.448902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527f8c-0ad2-4fde-9893-7a41b40d09b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:42.449331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a527f8c-0ad2-4fde-9893-7a41b40d09b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:42.457612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 953/1919] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:42.458546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757583f-1b39-4868-b2a8-d0d0bacfc37a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:42.460485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a527f8c-0ad2-4fde-9893-7a41b40d09b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:42.461948 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a527f8c-0ad2-4fde-9893-7a41b40d09b6 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:42.472982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757583f-1b39-4868-b2a8-d0d0bacfc37a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:42.473738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 967/1920] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:26:42.490512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:42.493142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:42.493671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:42.494084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] PATH_INFO: `/v3/projects/e9734df9b60e4e239c2df914a70c5c7f/users/8146247ce06e4730b9d3169ec263f786/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:42.513594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9734df9b60e4e239c2df914a70c5c7f, user_id=8146247ce06e4730b9d3169ec263f786, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:42.517882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:42.641883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 954/1921] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:42 2022] PUT /identity/v3/projects/e9734df9b60e4e239c2df914a70c5c7f/users/8146247ce06e4730b9d3169ec263f786/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:42.654402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187486f8-fcfc-4905-946e-c020909c12b9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:42.654852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187486f8-fcfc-4905-946e-c020909c12b9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:42.655241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187486f8-fcfc-4905-946e-c020909c12b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:42.687680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-187486f8-fcfc-4905-946e-c020909c12b9 None None] MFA Rules not processed for user `8146247ce06e4730b9d3169ec263f786`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:42.689282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-187486f8-fcfc-4905-946e-c020909c12b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:42.770031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 968/1922] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:42.781926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:42.783669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:42.885677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e7ce6-fb76-4cbc-ae9f-d6104f02c233 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:42.890146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7f37f-a8aa-42e2-bae2-b9e6f5a8bdb6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e939bc73c40140cb89517e918722d154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:42.892037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7f37f-a8aa-42e2-bae2-b9e6f5a8bdb6 service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:42.892426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7f37f-a8aa-42e2-bae2-b9e6f5a8bdb6 service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:42.892903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7f37f-a8aa-42e2-bae2-b9e6f5a8bdb6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:42.904597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7f37f-a8aa-42e2-bae2-b9e6f5a8bdb6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:42.906714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7f37f-a8aa-42e2-bae2-b9e6f5a8bdb6 service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:42.920518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 955/1923] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:43.197878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187486f8-fcfc-4905-946e-c020909c12b9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:43.199403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-187486f8-fcfc-4905-946e-c020909c12b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:43.272074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187486f8-fcfc-4905-946e-c020909c12b9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:43.276593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:43.277460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:43.277746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:43.278022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] PATH_INFO: `/v3/projects/80913cca9b594267870c36e4a7308407` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:43.279226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80913cca9b594267870c36e4a7308407)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:43.281217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:43.531988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80913cca9b594267870c36e4a7308407', 'request_id': 'req-ad98e66f-344a-4d5e-a776-33046a082710'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:43.537262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80913cca9b594267870c36e4a7308407', 'request_id': 'req-ad98e66f-344a-4d5e-a776-33046a082710'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:43.652397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80913cca9b594267870c36e4a7308407', 'request_id': 'req-ad98e66f-344a-4d5e-a776-33046a082710'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:43.664484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] The token cache is being invalidate because project 80913cca9b594267870c36e4a7308407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:43.664782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:43.665477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:43.670488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 969/1924] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:43 2022] DELETE /identity/v3/projects/80913cca9b594267870c36e4a7308407 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:43.671071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7f37f-a8aa-42e2-bae2-b9e6f5a8bdb6 service glance] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:43.698946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7f37f-a8aa-42e2-bae2-b9e6f5a8bdb6 service glance] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:43.795186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:43.796119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:43.796397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:43.797821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] PATH_INFO: `/v3/users/c4ccdd588f264ec0ba080878e2b3d359` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:43.799123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ccdd588f264ec0ba080878e2b3d359)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:43.802002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:43.974398 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:44.060441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ccdd588f264ec0ba080878e2b3d359'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:44.061103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ccdd588f264ec0ba080878e2b3d359'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:44.061866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ccdd588f264ec0ba080878e2b3d359'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:44.072866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ccdd588f264ec0ba080878e2b3d359'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:44.083008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1cd4eb94-fcba-43da-9172-1fd2defa327b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ccdd588f264ec0ba080878e2b3d359'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:44.137449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:44.269446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 956/1925] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:43 2022] DELETE /identity/v3/users/c4ccdd588f264ec0ba080878e2b3d359 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:26:44.819215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:44.820884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:44.846659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98e66f-344a-4d5e-a776-33046a082710 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:44.939061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:44.940847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:44.941400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:44.941829 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] PATH_INFO: `/v3/projects/6191a563a555474e9d838929bddcb073` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:44.943348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6191a563a555474e9d838929bddcb073)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:44.945576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:45.095417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6191a563a555474e9d838929bddcb073', 'request_id': 'req-6d25d437-ae2b-498b-b107-cdbb39e9c05f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:45.103343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6191a563a555474e9d838929bddcb073', 'request_id': 'req-6d25d437-ae2b-498b-b107-cdbb39e9c05f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:45.233854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6191a563a555474e9d838929bddcb073', 'request_id': 'req-6d25d437-ae2b-498b-b107-cdbb39e9c05f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:45.257746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] The token cache is being invalidate because project 6191a563a555474e9d838929bddcb073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:45.258143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:45.258908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6d25d437-ae2b-498b-b107-cdbb39e9c05f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:45.266607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 970/1926] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:44 2022] DELETE /identity/v3/projects/6191a563a555474e9d838929bddcb073 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:45.269819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c663b0-e4ab-4769-b81c-f649fe706c3e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:45.270316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c663b0-e4ab-4769-b81c-f649fe706c3e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:45.270492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c663b0-e4ab-4769-b81c-f649fe706c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:45.291713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b8c663b0-e4ab-4769-b81c-f649fe706c3e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:45.293013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b8c663b0-e4ab-4769-b81c-f649fe706c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:45.372106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 957/1927] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:45.381641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7b3a-35d1-403c-81aa-9ab9ad1e9f64 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:45.381882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7b3a-35d1-403c-81aa-9ab9ad1e9f64 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:45.382289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadd7b3a-35d1-403c-81aa-9ab9ad1e9f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:45.406345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dadd7b3a-35d1-403c-81aa-9ab9ad1e9f64 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:45.407930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dadd7b3a-35d1-403c-81aa-9ab9ad1e9f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:45.416842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 971/1928] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:45.421416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c663b0-e4ab-4769-b81c-f649fe706c3e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:45.433130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c663b0-e4ab-4769-b81c-f649fe706c3e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:45.439062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:45.439874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:45.440723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:45.441066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:45.441893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:45.442281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:45.444351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:45.458799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 958/1929] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:45.466898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd7b3a-35d1-403c-81aa-9ab9ad1e9f64 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:45.477571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadd7b3a-35d1-403c-81aa-9ab9ad1e9f64 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:45.482700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f04a-96b4-4768-b175-1f75ace0c96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:45.483538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f04a-96b4-4768-b175-1f75ace0c96f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:45.483830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f04a-96b4-4768-b175-1f75ace0c96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:45.484149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9f04a-96b4-4768-b175-1f75ace0c96f admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:45.484858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9f04a-96b4-4768-b175-1f75ace0c96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:45.486976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9f04a-96b4-4768-b175-1f75ace0c96f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:45.592070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 972/1930] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:45 2022] POST /identity/v3/projects => generated 361 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:45.595431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:45.605794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dd8a4-7258-4403-a460-1cc79c26baf0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:45.664895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d4a86-bc89-4bea-8d20-79c5188ec804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:45.665575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d4a86-bc89-4bea-8d20-79c5188ec804 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:45.665863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d4a86-bc89-4bea-8d20-79c5188ec804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:45.666144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d4a86-bc89-4bea-8d20-79c5188ec804 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:45.666852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d4a86-bc89-4bea-8d20-79c5188ec804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:45.668842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d4a86-bc89-4bea-8d20-79c5188ec804 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:45.769084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 959/1931] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:45 2022] POST /identity/v3/users => generated 281 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:45.773549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f04a-96b4-4768-b175-1f75ace0c96f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:45.784425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9f04a-96b4-4768-b175-1f75ace0c96f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:45.789172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940ce7b-86be-44f9-94b5-e81a1f1d4223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:45.789863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940ce7b-86be-44f9-94b5-e81a1f1d4223 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:45.790147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940ce7b-86be-44f9-94b5-e81a1f1d4223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:45.790426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940ce7b-86be-44f9-94b5-e81a1f1d4223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:45.791231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940ce7b-86be-44f9-94b5-e81a1f1d4223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:45.793016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940ce7b-86be-44f9-94b5-e81a1f1d4223 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:45.806050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 973/1932] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:45.809841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d4a86-bc89-4bea-8d20-79c5188ec804 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:45.821333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d4a86-bc89-4bea-8d20-79c5188ec804 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:45.826369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edee530-4d33-41be-bd63-bdd3d8cfb26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:45.829175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee530-4d33-41be-bd63-bdd3d8cfb26b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:45.829551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee530-4d33-41be-bd63-bdd3d8cfb26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:45.829911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edee530-4d33-41be-bd63-bdd3d8cfb26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:45.830716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edee530-4d33-41be-bd63-bdd3d8cfb26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:45.832681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edee530-4d33-41be-bd63-bdd3d8cfb26b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:45.847686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 960/1933] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:26:45.849263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940ce7b-86be-44f9-94b5-e81a1f1d4223 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:45.861637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940ce7b-86be-44f9-94b5-e81a1f1d4223 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:45.867842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:45.869199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:45.869487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:45.880174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] PATH_INFO: `/v3/projects/625c5b2fc8ba413f83f6553c4ee35e0e/users/b2d156b78a6a49b28296c579cfc691f8/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:45.889189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625c5b2fc8ba413f83f6553c4ee35e0e, user_id=b2d156b78a6a49b28296c579cfc691f8, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:45.892201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:46.018433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 974/1934] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:45 2022] PUT /identity/v3/projects/625c5b2fc8ba413f83f6553c4ee35e0e/users/b2d156b78a6a49b28296c579cfc691f8/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:46.031398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce226f9-6bca-467c-8227-4d1488df9157 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:46.033895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce226f9-6bca-467c-8227-4d1488df9157 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:46.034257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce226f9-6bca-467c-8227-4d1488df9157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:46.085637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cce226f9-6bca-467c-8227-4d1488df9157 None None] MFA Rules not processed for user `b2d156b78a6a49b28296c579cfc691f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:46.087204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cce226f9-6bca-467c-8227-4d1488df9157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:46.187538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 961/1935] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:46.230022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:46.231752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:46.276301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:46.329596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762369a-8466-4b9b-823d-3945978d5e47 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:46.333489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feba56-1fff-46b1-9e09-ef0f72d9536b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:46.334264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feba56-1fff-46b1-9e09-ef0f72d9536b service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:46.334550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feba56-1fff-46b1-9e09-ef0f72d9536b service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:46.334843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feba56-1fff-46b1-9e09-ef0f72d9536b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:46.350685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feba56-1fff-46b1-9e09-ef0f72d9536b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:46.354352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feba56-1fff-46b1-9e09-ef0f72d9536b service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:46.389307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 975/1936] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:46.404408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce226f9-6bca-467c-8227-4d1488df9157 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:46.503105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce226f9-6bca-467c-8227-4d1488df9157 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:46.507830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:46.510226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:46.510226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:46.510226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] PATH_INFO: `/v3/users/b2d156b78a6a49b28296c579cfc691f8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:46.510710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d156b78a6a49b28296c579cfc691f8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:46.512637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:46.750830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d156b78a6a49b28296c579cfc691f8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:46.751429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d156b78a6a49b28296c579cfc691f8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:46.752120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d156b78a6a49b28296c579cfc691f8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:46.759163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d156b78a6a49b28296c579cfc691f8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:46.771164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d156b78a6a49b28296c579cfc691f8'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:46.885864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 962/1937] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:46 2022] DELETE /identity/v3/users/b2d156b78a6a49b28296c579cfc691f8 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:46.894599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f7313-a057-4ac9-ba9d-cbe0e87fd933 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:46.895200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f7313-a057-4ac9-ba9d-cbe0e87fd933 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:46.895409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f7313-a057-4ac9-ba9d-cbe0e87fd933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:46.911993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:46.919522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2b5f7313-a057-4ac9-ba9d-cbe0e87fd933 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:46.922387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2b5f7313-a057-4ac9-ba9d-cbe0e87fd933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:46.987782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 976/1938] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:46.998518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:47.000057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:47.056220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88be360-3ba5-49da-b3e7-81bf8f528505 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:47.059826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a00fe4-b9b8-4441-889b-573b3490d0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:47.060485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00fe4-b9b8-4441-889b-573b3490d0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:47.060771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00fe4-b9b8-4441-889b-573b3490d0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:47.061048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00fe4-b9b8-4441-889b-573b3490d0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:47.078030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a00fe4-b9b8-4441-889b-573b3490d0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:47.079529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a00fe4-b9b8-4441-889b-573b3490d0b3 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:47.097373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 963/1939] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:47.617455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f7313-a057-4ac9-ba9d-cbe0e87fd933 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:47.617455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2b5f7313-a057-4ac9-ba9d-cbe0e87fd933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:47.629190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f7313-a057-4ac9-ba9d-cbe0e87fd933 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:47.634058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:47.634930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:47.635280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:47.635665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] PATH_INFO: `/v3/projects/625c5b2fc8ba413f83f6553c4ee35e0e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:47.636952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625c5b2fc8ba413f83f6553c4ee35e0e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:47.638824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:47.693717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:47.776727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625c5b2fc8ba413f83f6553c4ee35e0e', 'request_id': 'req-20a779c7-4495-4430-a47e-589c1e54a2c0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:47.782729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625c5b2fc8ba413f83f6553c4ee35e0e', 'request_id': 'req-20a779c7-4495-4430-a47e-589c1e54a2c0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:47.902598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625c5b2fc8ba413f83f6553c4ee35e0e', 'request_id': 'req-20a779c7-4495-4430-a47e-589c1e54a2c0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:47.915090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] The token cache is being invalidate because project 625c5b2fc8ba413f83f6553c4ee35e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:47.915542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:47.916198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-20a779c7-4495-4430-a47e-589c1e54a2c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:47.917879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 977/1940] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:47 2022] DELETE /identity/v3/projects/625c5b2fc8ba413f83f6553c4ee35e0e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:47.922753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeeaea-041c-45f7-8398-b9ff9705927f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:47.923344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeeaea-041c-45f7-8398-b9ff9705927f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:47.923451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeeaea-041c-45f7-8398-b9ff9705927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:47.941559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-43aeeaea-041c-45f7-8398-b9ff9705927f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:47.942764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-43aeeaea-041c-45f7-8398-b9ff9705927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:48.009735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 964/1941] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:48.019448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124fe3-c65f-4f8a-90d3-32543712f941 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.019969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124fe3-c65f-4f8a-90d3-32543712f941 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.020496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3124fe3-c65f-4f8a-90d3-32543712f941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.046635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b3124fe3-c65f-4f8a-90d3-32543712f941 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:48.049015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b3124fe3-c65f-4f8a-90d3-32543712f941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:48.065772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 978/1942] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:48.068977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeeaea-041c-45f7-8398-b9ff9705927f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:48.082326 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeeaea-041c-45f7-8398-b9ff9705927f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:48.089096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:48.089823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.090108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.090398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.091243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:48.091594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:48.093404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:48.108071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 965/1943] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:26:48.112353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3124fe3-c65f-4f8a-90d3-32543712f941 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:48.123186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3124fe3-c65f-4f8a-90d3-32543712f941 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:48.130990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce53a89-6be9-4c83-a005-0f34d1294a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:48.139916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce53a89-6be9-4c83-a005-0f34d1294a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.139916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce53a89-6be9-4c83-a005-0f34d1294a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.139916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce53a89-6be9-4c83-a005-0f34d1294a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.139916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce53a89-6be9-4c83-a005-0f34d1294a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:48.139916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce53a89-6be9-4c83-a005-0f34d1294a04 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:48.248424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 979/1944] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:48 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:48.252276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:48.263968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b74833-69e5-47a6-b10e-a7e2aa6c383d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:48.321065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:48.324512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf318b-f19d-4a3a-ba0f-80fe423cc72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:48.325247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf318b-f19d-4a3a-ba0f-80fe423cc72e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.325535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf318b-f19d-4a3a-ba0f-80fe423cc72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.325814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf318b-f19d-4a3a-ba0f-80fe423cc72e admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.326512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf318b-f19d-4a3a-ba0f-80fe423cc72e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:48.344242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf318b-f19d-4a3a-ba0f-80fe423cc72e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:48.437590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 966/1945] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:48 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:48.439286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce53a89-6be9-4c83-a005-0f34d1294a04 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:48.452313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce53a89-6be9-4c83-a005-0f34d1294a04 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:48.461574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842b457-937b-4c93-a926-8002438eb151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:48.462563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b457-937b-4c93-a926-8002438eb151 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.462852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b457-937b-4c93-a926-8002438eb151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.466646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842b457-937b-4c93-a926-8002438eb151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.466646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842b457-937b-4c93-a926-8002438eb151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:48.466646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842b457-937b-4c93-a926-8002438eb151 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:48.481075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 980/1946] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:48.482536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf318b-f19d-4a3a-ba0f-80fe423cc72e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:48.495634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf318b-f19d-4a3a-ba0f-80fe423cc72e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:48.501096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e94b5-652a-4f87-b01d-90b32f3522b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:48.501940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e94b5-652a-4f87-b01d-90b32f3522b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.502231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e94b5-652a-4f87-b01d-90b32f3522b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.502509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e94b5-652a-4f87-b01d-90b32f3522b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.503293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e94b5-652a-4f87-b01d-90b32f3522b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:48.505104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e94b5-652a-4f87-b01d-90b32f3522b7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:48.520024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 967/1947] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:48.522200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842b457-937b-4c93-a926-8002438eb151 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:48.534014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842b457-937b-4c93-a926-8002438eb151 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:48.545586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:48.547069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.551362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.551720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] PATH_INFO: `/v3/projects/bf718c5d8a06478dbf7254526580dbe3/users/98157ef62d924ed28fa6b1adba4c00fb/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.569602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf718c5d8a06478dbf7254526580dbe3, user_id=98157ef62d924ed28fa6b1adba4c00fb, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:48.575916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:48.697233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 981/1948] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:48 2022] PUT /identity/v3/projects/bf718c5d8a06478dbf7254526580dbe3/users/98157ef62d924ed28fa6b1adba4c00fb/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:26:48.708205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fa4b5-4d21-4d93-89ca-3d85cfa2f92e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.708626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fa4b5-4d21-4d93-89ca-3d85cfa2f92e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.709008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fa4b5-4d21-4d93-89ca-3d85cfa2f92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.742441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-651fa4b5-4d21-4d93-89ca-3d85cfa2f92e None None] MFA Rules not processed for user `98157ef62d924ed28fa6b1adba4c00fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:48.743928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-651fa4b5-4d21-4d93-89ca-3d85cfa2f92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:48.841020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 968/1949] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:48.853567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:48.856750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:48.931671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0b2f4-55b4-418d-9ca4-b0e60aa538c1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:48.935438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74466caa-6fc3-4139-ad4d-d743f346c301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:48.936151 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74466caa-6fc3-4139-ad4d-d743f346c301 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:48.936440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74466caa-6fc3-4139-ad4d-d743f346c301 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:48.936719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74466caa-6fc3-4139-ad4d-d743f346c301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:48.945811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74466caa-6fc3-4139-ad4d-d743f346c301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:48.947420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74466caa-6fc3-4139-ad4d-d743f346c301 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:48.957368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 982/1950] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:49.622654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:49.903846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fa4b5-4d21-4d93-89ca-3d85cfa2f92e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:49.906512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-651fa4b5-4d21-4d93-89ca-3d85cfa2f92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:49.981187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fa4b5-4d21-4d93-89ca-3d85cfa2f92e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:49.987798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:49.988675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:49.989082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:49.989379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] PATH_INFO: `/v3/users/5504e395d523412c9626e57a99939f3d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:49.990628 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5504e395d523412c9626e57a99939f3d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:49.993006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:50.178598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5504e395d523412c9626e57a99939f3d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:50.179305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5504e395d523412c9626e57a99939f3d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:50.179888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5504e395d523412c9626e57a99939f3d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:50.186889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5504e395d523412c9626e57a99939f3d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:50.198999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5504e395d523412c9626e57a99939f3d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:50.234455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:50.267493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 969/1951] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:49 2022] DELETE /identity/v3/users/5504e395d523412c9626e57a99939f3d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:50.272042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e084f-f55a-4a34-a8c9-a78026380073 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:50.272529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e084f-f55a-4a34-a8c9-a78026380073 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:50.272635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e084f-f55a-4a34-a8c9-a78026380073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:50.289718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-456e084f-f55a-4a34-a8c9-a78026380073 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:50.290914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-456e084f-f55a-4a34-a8c9-a78026380073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:50.352288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 983/1952] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:26:50.362360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:50.363908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:50.420975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091ce75-44b6-4765-b420-f7a54a3c2745 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:50.425121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe2d60-7c39-4a0b-a1b8-5e1d4ff2d08e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:50.425763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe2d60-7c39-4a0b-a1b8-5e1d4ff2d08e service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:50.426299 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe2d60-7c39-4a0b-a1b8-5e1d4ff2d08e service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:50.426572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe2d60-7c39-4a0b-a1b8-5e1d4ff2d08e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:50.436060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe2d60-7c39-4a0b-a1b8-5e1d4ff2d08e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:50.438085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe2d60-7c39-4a0b-a1b8-5e1d4ff2d08e service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:50.447765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 970/1953] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:50.957112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e084f-f55a-4a34-a8c9-a78026380073 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:50.958634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-456e084f-f55a-4a34-a8c9-a78026380073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:50.997796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e084f-f55a-4a34-a8c9-a78026380073 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:51.003083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:51.004111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.004836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.005126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] PATH_INFO: `/v3/projects/ebc6d477259b4c4880e64819918ffbbc` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.006586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc6d477259b4c4880e64819918ffbbc)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:51.008594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:51.176261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc6d477259b4c4880e64819918ffbbc', 'request_id': 'req-91bb7432-31aa-46a4-814a-0c59fd911a52'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:51.184129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc6d477259b4c4880e64819918ffbbc', 'request_id': 'req-91bb7432-31aa-46a4-814a-0c59fd911a52'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:51.262368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc6d477259b4c4880e64819918ffbbc', 'request_id': 'req-91bb7432-31aa-46a4-814a-0c59fd911a52'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:51.274943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] The token cache is being invalidate because project ebc6d477259b4c4880e64819918ffbbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:51.275529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:51.275984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-91bb7432-31aa-46a4-814a-0c59fd911a52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:51.277586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 984/1954] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:50 2022] DELETE /identity/v3/projects/ebc6d477259b4c4880e64819918ffbbc => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:51.283051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc347-db53-4db8-95f8-5769809e3d5e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.283746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc347-db53-4db8-95f8-5769809e3d5e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.283746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcc347-db53-4db8-95f8-5769809e3d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.302260 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c7bcc347-db53-4db8-95f8-5769809e3d5e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:51.303505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c7bcc347-db53-4db8-95f8-5769809e3d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:51.368702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 971/1955] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:51.375851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:51.378185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91e4fb-8f55-48fe-9509-0c55fe684083 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.378477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91e4fb-8f55-48fe-9509-0c55fe684083 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.378758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91e4fb-8f55-48fe-9509-0c55fe684083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.401367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8e91e4fb-8f55-48fe-9509-0c55fe684083 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:51.403094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8e91e4fb-8f55-48fe-9509-0c55fe684083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:51.412041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 985/1956] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:51.416495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcc347-db53-4db8-95f8-5769809e3d5e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:51.426518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcc347-db53-4db8-95f8-5769809e3d5e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:51.431091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:51.431768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.432049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.432324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.433042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:51.433346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:51.434993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:51.447685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 972/1957] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:26:51.451926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91e4fb-8f55-48fe-9509-0c55fe684083 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:51.462814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91e4fb-8f55-48fe-9509-0c55fe684083 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:51.467621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be808ba-e92c-4dec-8b96-e239196fe16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:51.468303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be808ba-e92c-4dec-8b96-e239196fe16c admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.468596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be808ba-e92c-4dec-8b96-e239196fe16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.468897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be808ba-e92c-4dec-8b96-e239196fe16c admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.469579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be808ba-e92c-4dec-8b96-e239196fe16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:51.471512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be808ba-e92c-4dec-8b96-e239196fe16c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:51.560898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 986/1958] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:51.565508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:51.575342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea91d80-6802-46e9-916d-6091c8013de5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:51.632959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1daa98c-d13b-45f2-901f-a7e8c944201c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:51.633634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1daa98c-d13b-45f2-901f-a7e8c944201c admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.633918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1daa98c-d13b-45f2-901f-a7e8c944201c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.634195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1daa98c-d13b-45f2-901f-a7e8c944201c admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.634887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1daa98c-d13b-45f2-901f-a7e8c944201c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:51.636835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1daa98c-d13b-45f2-901f-a7e8c944201c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:51.748666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 973/1959] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:51 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:51.753205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be808ba-e92c-4dec-8b96-e239196fe16c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:51.766455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be808ba-e92c-4dec-8b96-e239196fe16c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:51.771370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaade6bb-f4bf-429b-a08a-631b7e2f0818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:51.772043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaade6bb-f4bf-429b-a08a-631b7e2f0818 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.772332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaade6bb-f4bf-429b-a08a-631b7e2f0818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.772609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaade6bb-f4bf-429b-a08a-631b7e2f0818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.773528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaade6bb-f4bf-429b-a08a-631b7e2f0818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:51.775160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaade6bb-f4bf-429b-a08a-631b7e2f0818 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:51.789088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 987/1960] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:51.793473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1daa98c-d13b-45f2-901f-a7e8c944201c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:51.805604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1daa98c-d13b-45f2-901f-a7e8c944201c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:51.810539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fafa0-9816-4e3f-a8dd-c7950dd2eafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:51.811179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fafa0-9816-4e3f-a8dd-c7950dd2eafb admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.811497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fafa0-9816-4e3f-a8dd-c7950dd2eafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.811779 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fafa0-9816-4e3f-a8dd-c7950dd2eafb admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.812548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fafa0-9816-4e3f-a8dd-c7950dd2eafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:51.814141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fafa0-9816-4e3f-a8dd-c7950dd2eafb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:51.824565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 974/1961] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:51.834275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaade6bb-f4bf-429b-a08a-631b7e2f0818 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:51.844842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaade6bb-f4bf-429b-a08a-631b7e2f0818 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:51.849410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5572aa7-e589-44f3-baff-f281600a96f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:51.850298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5572aa7-e589-44f3-baff-f281600a96f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.854832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5572aa7-e589-44f3-baff-f281600a96f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.854832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5572aa7-e589-44f3-baff-f281600a96f3 admin admin] PATH_INFO: `/v3/projects/9c6d5f015a4142c597509d1c48dc4a31/users/0b29c5b4c6c84a6791fde567d98b52fc/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:51.865464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5572aa7-e589-44f3-baff-f281600a96f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6d5f015a4142c597509d1c48dc4a31, user_id=0b29c5b4c6c84a6791fde567d98b52fc, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:51.868154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5572aa7-e589-44f3-baff-f281600a96f3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:51.972853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 988/1962] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:51 2022] PUT /identity/v3/projects/9c6d5f015a4142c597509d1c48dc4a31/users/0b29c5b4c6c84a6791fde567d98b52fc/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:51.984722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdb8fd-8488-4d94-b365-7f36c24aa293 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:51.985034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdb8fd-8488-4d94-b365-7f36c24aa293 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:51.985319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdb8fd-8488-4d94-b365-7f36c24aa293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.011903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-66cdb8fd-8488-4d94-b365-7f36c24aa293 None None] MFA Rules not processed for user `0b29c5b4c6c84a6791fde567d98b52fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:52.013126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-66cdb8fd-8488-4d94-b365-7f36c24aa293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:52.080898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 975/1963] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:52.086074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5572aa7-e589-44f3-baff-f281600a96f3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:52.096982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:52.151884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5572aa7-e589-44f3-baff-f281600a96f3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:52.156957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496ce25-9bc0-4be9-8e15-bf86b954243c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:52.160601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496ce25-9bc0-4be9-8e15-bf86b954243c admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:52.160601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496ce25-9bc0-4be9-8e15-bf86b954243c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:52.160601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496ce25-9bc0-4be9-8e15-bf86b954243c admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.160601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496ce25-9bc0-4be9-8e15-bf86b954243c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:52.161025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496ce25-9bc0-4be9-8e15-bf86b954243c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:52.264887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 989/1964] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:52 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:52.269514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdb8fd-8488-4d94-b365-7f36c24aa293 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:52.280287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdb8fd-8488-4d94-b365-7f36c24aa293 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:52.338507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49741399-787f-4bd0-92e2-d304d1021139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:52.339170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49741399-787f-4bd0-92e2-d304d1021139 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:52.339493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49741399-787f-4bd0-92e2-d304d1021139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:52.339777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49741399-787f-4bd0-92e2-d304d1021139 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.340532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49741399-787f-4bd0-92e2-d304d1021139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:52.342603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49741399-787f-4bd0-92e2-d304d1021139 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:52.463194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 976/1965] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:52 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:52.465821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496ce25-9bc0-4be9-8e15-bf86b954243c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:52.476927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496ce25-9bc0-4be9-8e15-bf86b954243c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:52.482986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bace71-f88e-4d7a-9445-a78fb123509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:52.485354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bace71-f88e-4d7a-9445-a78fb123509e admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:52.485354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bace71-f88e-4d7a-9445-a78fb123509e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:52.485354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bace71-f88e-4d7a-9445-a78fb123509e admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.485640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bace71-f88e-4d7a-9445-a78fb123509e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:52.487259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bace71-f88e-4d7a-9445-a78fb123509e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:52.503039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 990/1966] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:52.504278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49741399-787f-4bd0-92e2-d304d1021139 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:52.516753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49741399-787f-4bd0-92e2-d304d1021139 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:52.521602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02775f68-79a9-40a6-bbe9-3f1c248f605d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:52.522505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02775f68-79a9-40a6-bbe9-3f1c248f605d admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:52.522770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02775f68-79a9-40a6-bbe9-3f1c248f605d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:52.523046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02775f68-79a9-40a6-bbe9-3f1c248f605d admin admin] PATH_INFO: `/v3/projects/e115bf3fe26b40a882cffcd0701c4d4f/users/c03859aa46254b359228adb29edbfaac/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.531288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:52.537685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02775f68-79a9-40a6-bbe9-3f1c248f605d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e115bf3fe26b40a882cffcd0701c4d4f, user_id=c03859aa46254b359228adb29edbfaac, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:52.540391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02775f68-79a9-40a6-bbe9-3f1c248f605d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:52.622023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 977/1967] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:52 2022] PUT /identity/v3/projects/e115bf3fe26b40a882cffcd0701c4d4f/users/c03859aa46254b359228adb29edbfaac/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:52.626894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bace71-f88e-4d7a-9445-a78fb123509e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:52.636805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bace71-f88e-4d7a-9445-a78fb123509e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:52.692543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602bca3-38d1-4331-b692-957726082de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:52.693214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602bca3-38d1-4331-b692-957726082de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:52.693502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602bca3-38d1-4331-b692-957726082de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:52.693782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602bca3-38d1-4331-b692-957726082de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.694543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602bca3-38d1-4331-b692-957726082de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:52.696167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602bca3-38d1-4331-b692-957726082de8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:52.705723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 991/1968] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:52.709897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02775f68-79a9-40a6-bbe9-3f1c248f605d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:52.720540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02775f68-79a9-40a6-bbe9-3f1c248f605d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:52.724783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:52.725654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:52.725934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:52.726268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] PATH_INFO: `/v3/projects/e115bf3fe26b40a882cffcd0701c4d4f/users/c03859aa46254b359228adb29edbfaac/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.728228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e115bf3fe26b40a882cffcd0701c4d4f, user_id=c03859aa46254b359228adb29edbfaac, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:52.730839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:52.752911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 978/1969] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:52 2022] PUT /identity/v3/projects/e115bf3fe26b40a882cffcd0701c4d4f/users/c03859aa46254b359228adb29edbfaac/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:52.766267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a60fd-55b1-43f7-813e-edddcf1840ad None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:52.766539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a60fd-55b1-43f7-813e-edddcf1840ad None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:52.766821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a60fd-55b1-43f7-813e-edddcf1840ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.796007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-df8a60fd-55b1-43f7-813e-edddcf1840ad None None] MFA Rules not processed for user `c03859aa46254b359228adb29edbfaac`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:52.797804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-df8a60fd-55b1-43f7-813e-edddcf1840ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:52.863853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 992/1970] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:52.874560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:52.876264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:52.932806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adeb8c-ebe7-45cf-9ffd-3e569eff6581 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:52.936342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8b6ef-7e81-4d46-b709-75bb1c4143a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:52.936981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8b6ef-7e81-4d46-b709-75bb1c4143a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:52.937265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8b6ef-7e81-4d46-b709-75bb1c4143a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:52.937540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8b6ef-7e81-4d46-b709-75bb1c4143a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:52.993695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8b6ef-7e81-4d46-b709-75bb1c4143a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:52.995036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8b6ef-7e81-4d46-b709-75bb1c4143a5 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:53.004096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 979/1971] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:53.230684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a60fd-55b1-43f7-813e-edddcf1840ad None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:53.232107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-df8a60fd-55b1-43f7-813e-edddcf1840ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:53.298338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a60fd-55b1-43f7-813e-edddcf1840ad None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:53.303564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:53.304442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:53.304731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:53.305023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] PATH_INFO: `/v3/users/98157ef62d924ed28fa6b1adba4c00fb` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:53.306208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98157ef62d924ed28fa6b1adba4c00fb)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:53.308147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:53.564730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98157ef62d924ed28fa6b1adba4c00fb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:53.565459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98157ef62d924ed28fa6b1adba4c00fb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:53.566205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98157ef62d924ed28fa6b1adba4c00fb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:53.573563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98157ef62d924ed28fa6b1adba4c00fb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:53.585876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98157ef62d924ed28fa6b1adba4c00fb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:53.673644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 993/1972] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:53 2022] DELETE /identity/v3/users/98157ef62d924ed28fa6b1adba4c00fb => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:53.678459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8257b-3e93-4f86-983e-702a7c0f8274 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:53.678940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8257b-3e93-4f86-983e-702a7c0f8274 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:53.679060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8257b-3e93-4f86-983e-702a7c0f8274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:53.698307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-46b8257b-3e93-4f86-983e-702a7c0f8274 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:53.699571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-46b8257b-3e93-4f86-983e-702a7c0f8274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:53.762823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 980/1973] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:53.773386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:53.779286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:53.847668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729f4ff-54d3-4deb-843d-b6eb5ef49057 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:53.851296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6c6d4-8223-4346-9f87-36e05d58b655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:53.851954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6c6d4-8223-4346-9f87-36e05d58b655 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:53.852239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6c6d4-8223-4346-9f87-36e05d58b655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:53.852518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6c6d4-8223-4346-9f87-36e05d58b655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:53.862046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6c6d4-8223-4346-9f87-36e05d58b655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:53.868025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6c6d4-8223-4346-9f87-36e05d58b655 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:53.876437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 994/1974] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:54.439435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:54.481122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8257b-3e93-4f86-983e-702a7c0f8274 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:54.482600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-46b8257b-3e93-4f86-983e-702a7c0f8274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:54.496102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8257b-3e93-4f86-983e-702a7c0f8274 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:54.500905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:54.501780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:54.502059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:54.502895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] PATH_INFO: `/v3/projects/bf718c5d8a06478dbf7254526580dbe3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:54.503570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf718c5d8a06478dbf7254526580dbe3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:54.507037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:54.548926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:54.628736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf718c5d8a06478dbf7254526580dbe3', 'request_id': 'req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:54.634400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf718c5d8a06478dbf7254526580dbe3', 'request_id': 'req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:54.804898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf718c5d8a06478dbf7254526580dbe3', 'request_id': 'req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:54.816978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] The token cache is being invalidate because project bf718c5d8a06478dbf7254526580dbe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:54.817274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:54.817911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b2bbda52-6e3d-41fa-b5a2-9d31df4fae88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:54.819485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 981/1975] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:54 2022] DELETE /identity/v3/projects/bf718c5d8a06478dbf7254526580dbe3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:54.825297 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3ff3-85c1-4285-9507-3f9202a9805d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:54.825790 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3ff3-85c1-4285-9507-3f9202a9805d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:54.825922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb3ff3-85c1-4285-9507-3f9202a9805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:54.845024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4ddb3ff3-85c1-4285-9507-3f9202a9805d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:54.846241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4ddb3ff3-85c1-4285-9507-3f9202a9805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:54.917449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 995/1976] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:54.925596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820cc9f-9a3d-48fe-8bb2-11b35af5a8c1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:54.926146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820cc9f-9a3d-48fe-8bb2-11b35af5a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:54.926463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820cc9f-9a3d-48fe-8bb2-11b35af5a8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:54.945055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7820cc9f-9a3d-48fe-8bb2-11b35af5a8c1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:54.946309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7820cc9f-9a3d-48fe-8bb2-11b35af5a8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:54.955886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 982/1977] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:54.960204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb3ff3-85c1-4285-9507-3f9202a9805d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:54.970996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb3ff3-85c1-4285-9507-3f9202a9805d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:54.975758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:54.978090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:54.978090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:54.978090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:54.978090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:54.978090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:54.979996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:54.992674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 996/1978] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:26:54.997795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820cc9f-9a3d-48fe-8bb2-11b35af5a8c1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:55.008076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820cc9f-9a3d-48fe-8bb2-11b35af5a8c1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:55.012912 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e3f60-25cf-4210-9f28-ba7262bb8155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:55.013778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e3f60-25cf-4210-9f28-ba7262bb8155 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:55.014224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e3f60-25cf-4210-9f28-ba7262bb8155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:55.014507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e3f60-25cf-4210-9f28-ba7262bb8155 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:55.015868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e3f60-25cf-4210-9f28-ba7262bb8155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:55.017797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e3f60-25cf-4210-9f28-ba7262bb8155 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:55.122643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 983/1979] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:54 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:55.127479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:55.138483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be3ec6-842d-4e6e-83ac-7fbd1072d7a9 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:55.200597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b9d8-c19f-458f-98ab-398ca2d688d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:55.201368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b9d8-c19f-458f-98ab-398ca2d688d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:55.201654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b9d8-c19f-458f-98ab-398ca2d688d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:55.201939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50b9d8-c19f-458f-98ab-398ca2d688d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:55.203310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b9d8-c19f-458f-98ab-398ca2d688d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:55.206888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50b9d8-c19f-458f-98ab-398ca2d688d8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:55.285700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 997/1980] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:55 2022] POST /identity/v3/users => generated 283 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:26:55.289913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e3f60-25cf-4210-9f28-ba7262bb8155 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:55.301697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e3f60-25cf-4210-9f28-ba7262bb8155 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:55.307398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f8c80-11f6-43b6-975b-017b157039f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:55.308066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8c80-11f6-43b6-975b-017b157039f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:55.308348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8c80-11f6-43b6-975b-017b157039f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:55.308623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f8c80-11f6-43b6-975b-017b157039f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:55.309350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f8c80-11f6-43b6-975b-017b157039f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:55.310939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f8c80-11f6-43b6-975b-017b157039f5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:55.322060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 984/1981] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:55.326905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b9d8-c19f-458f-98ab-398ca2d688d8 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:55.338463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50b9d8-c19f-458f-98ab-398ca2d688d8 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:55.344082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee5317-76c1-497f-8018-296936756844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:55.344474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee5317-76c1-497f-8018-296936756844 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:55.344601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee5317-76c1-497f-8018-296936756844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:55.344879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee5317-76c1-497f-8018-296936756844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:55.345818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee5317-76c1-497f-8018-296936756844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:55.347356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee5317-76c1-497f-8018-296936756844 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:55.359914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 998/1982] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:55.365770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f8c80-11f6-43b6-975b-017b157039f5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:55.376937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f8c80-11f6-43b6-975b-017b157039f5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:55.382176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:55.383046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:55.383483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:55.383811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] PATH_INFO: `/v3/projects/960d84c221fb4a1eabc3a6d1ae34a97e/users/fd8707405a044f0a991a3129739f1207/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:55.400006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=960d84c221fb4a1eabc3a6d1ae34a97e, user_id=fd8707405a044f0a991a3129739f1207, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:55.402699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:55.486040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 985/1983] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:55 2022] PUT /identity/v3/projects/960d84c221fb4a1eabc3a6d1ae34a97e/users/fd8707405a044f0a991a3129739f1207/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:55.496415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0488f-0bfa-434d-87c6-70c9b1f258d8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:55.496714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0488f-0bfa-434d-87c6-70c9b1f258d8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:55.497020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0488f-0bfa-434d-87c6-70c9b1f258d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:55.525552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b0d0488f-0bfa-434d-87c6-70c9b1f258d8 None None] MFA Rules not processed for user `fd8707405a044f0a991a3129739f1207`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:55.526811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b0d0488f-0bfa-434d-87c6-70c9b1f258d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:55.596685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 999/1984] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:55.609337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:55.610671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:55.674607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c2aaf-9f64-4c4e-8b9d-fddb6ee02a8b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:55.678963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfec764-709e-4ef6-9bd0-efd2018c7bb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:55.679663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfec764-709e-4ef6-9bd0-efd2018c7bb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:55.679953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfec764-709e-4ef6-9bd0-efd2018c7bb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:55.680231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfec764-709e-4ef6-9bd0-efd2018c7bb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:55.689812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfec764-709e-4ef6-9bd0-efd2018c7bb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:55.691242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfec764-709e-4ef6-9bd0-efd2018c7bb5 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:55.700480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 986/1985] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:56.102564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:56.177660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0488f-0bfa-434d-87c6-70c9b1f258d8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:56.179113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b0d0488f-0bfa-434d-87c6-70c9b1f258d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:56.263236 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0488f-0bfa-434d-87c6-70c9b1f258d8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:56.277105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:56.278009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:56.278290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:56.278564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] PATH_INFO: `/v3/users/0b29c5b4c6c84a6791fde567d98b52fc` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:56.279850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b29c5b4c6c84a6791fde567d98b52fc)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:56.282550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:56.478823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b29c5b4c6c84a6791fde567d98b52fc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:56.480755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b29c5b4c6c84a6791fde567d98b52fc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:56.481550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b29c5b4c6c84a6791fde567d98b52fc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:56.490735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b29c5b4c6c84a6791fde567d98b52fc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:56.513277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b29c5b4c6c84a6791fde567d98b52fc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:56.670847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1000/1986] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:56 2022] DELETE /identity/v3/users/0b29c5b4c6c84a6791fde567d98b52fc => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:56.675437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031db1b-d890-411d-8902-f1db0ce09553 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:56.676172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031db1b-d890-411d-8902-f1db0ce09553 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:56.676292 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031db1b-d890-411d-8902-f1db0ce09553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:56.693728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0031db1b-d890-411d-8902-f1db0ce09553 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:56.694930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0031db1b-d890-411d-8902-f1db0ce09553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:56.751526 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 987/1987] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:56.761363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:56.768410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:56.824012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b0e3-4a49-4f68-8ccd-aa8f0fe8c73c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:56.830728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52788e9f-c1d7-40a4-8b6c-a6e04579671b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:56.848573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52788e9f-c1d7-40a4-8b6c-a6e04579671b service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:56.848573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52788e9f-c1d7-40a4-8b6c-a6e04579671b service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:56.848573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52788e9f-c1d7-40a4-8b6c-a6e04579671b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:56.848573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52788e9f-c1d7-40a4-8b6c-a6e04579671b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:56.848573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52788e9f-c1d7-40a4-8b6c-a6e04579671b service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:56.860432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1001/1988] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:26:56.908566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:57.332039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031db1b-d890-411d-8902-f1db0ce09553 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:57.333565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0031db1b-d890-411d-8902-f1db0ce09553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:57.348916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031db1b-d890-411d-8902-f1db0ce09553 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:57.355116 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:57.358076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:57.358304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:57.358629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] PATH_INFO: `/v3/projects/9c6d5f015a4142c597509d1c48dc4a31` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:57.360681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6d5f015a4142c597509d1c48dc4a31)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:57.362842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:57.451392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6d5f015a4142c597509d1c48dc4a31', 'request_id': 'req-be811f2a-89db-4473-a94c-995e25019338'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.456782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6d5f015a4142c597509d1c48dc4a31', 'request_id': 'req-be811f2a-89db-4473-a94c-995e25019338'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.519532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6d5f015a4142c597509d1c48dc4a31', 'request_id': 'req-be811f2a-89db-4473-a94c-995e25019338'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.540358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] The token cache is being invalidate because project 9c6d5f015a4142c597509d1c48dc4a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:57.540358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.540358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.540358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 988/1989] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:57 2022] DELETE /identity/v3/projects/9c6d5f015a4142c597509d1c48dc4a31 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:57.542086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52788e9f-c1d7-40a4-8b6c-a6e04579671b service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:57.553608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52788e9f-c1d7-40a4-8b6c-a6e04579671b service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:57.613776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:57.615792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:57.628923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:57.629227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] PATH_INFO: `/v3/users/c03859aa46254b359228adb29edbfaac` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:57.632396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03859aa46254b359228adb29edbfaac)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:57.637708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:57.860068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03859aa46254b359228adb29edbfaac'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.860749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03859aa46254b359228adb29edbfaac'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.861404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03859aa46254b359228adb29edbfaac'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.870359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03859aa46254b359228adb29edbfaac'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.873752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:57.884666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2dfafc49-de10-4042-a105-d8afa59e807d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03859aa46254b359228adb29edbfaac'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:57.999240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1002/1990] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:26:57 2022] DELETE /identity/v3/users/c03859aa46254b359228adb29edbfaac => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:58.469536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:58.470978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:58.518489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be811f2a-89db-4473-a94c-995e25019338 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:58.594749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:58.596108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:58.596457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:58.597491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] PATH_INFO: `/v3/projects/e115bf3fe26b40a882cffcd0701c4d4f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:58.600713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:58.601807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e115bf3fe26b40a882cffcd0701c4d4f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:58.603936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:58.795658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e115bf3fe26b40a882cffcd0701c4d4f', 'request_id': 'req-d16c069d-0303-4ad0-806b-dbbcf8ffd782'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:58.803404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e115bf3fe26b40a882cffcd0701c4d4f', 'request_id': 'req-d16c069d-0303-4ad0-806b-dbbcf8ffd782'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:58.862028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e115bf3fe26b40a882cffcd0701c4d4f', 'request_id': 'req-d16c069d-0303-4ad0-806b-dbbcf8ffd782'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:58.873470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] The token cache is being invalidate because project e115bf3fe26b40a882cffcd0701c4d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:26:58.873791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:58.874419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d16c069d-0303-4ad0-806b-dbbcf8ffd782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:26:58.876514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 989/1991] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:26:58 2022] DELETE /identity/v3/projects/e115bf3fe26b40a882cffcd0701c4d4f => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:58.881443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b433d4-14d2-475f-aa3f-3d4b82ea1467 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:58.881951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b433d4-14d2-475f-aa3f-3d4b82ea1467 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:58.882053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b433d4-14d2-475f-aa3f-3d4b82ea1467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:58.900634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-69b433d4-14d2-475f-aa3f-3d4b82ea1467 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:58.901844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-69b433d4-14d2-475f-aa3f-3d4b82ea1467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:58.959365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1003/1992] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:58.968201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f824-0bb5-4205-81cc-ca71cdc51f4c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:58.968533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f824-0bb5-4205-81cc-ca71cdc51f4c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:58.968832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f824-0bb5-4205-81cc-ca71cdc51f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:58.988782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c7a7f824-0bb5-4205-81cc-ca71cdc51f4c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:58.989946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c7a7f824-0bb5-4205-81cc-ca71cdc51f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:58.997846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 990/1993] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:59.002566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b433d4-14d2-475f-aa3f-3d4b82ea1467 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:59.012411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b433d4-14d2-475f-aa3f-3d4b82ea1467 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:59.018300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:59.018964 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:59.029445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:59.029445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:59.029445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:26:59.029445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:59.029445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:59.033736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1004/1994] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:26:59.037987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7f824-0bb5-4205-81cc-ca71cdc51f4c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:59.047096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7f824-0bb5-4205-81cc-ca71cdc51f4c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:59.052005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984fdce-ca6f-4ffd-be56-85ca70f2146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:59.052719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984fdce-ca6f-4ffd-be56-85ca70f2146a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:59.053002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984fdce-ca6f-4ffd-be56-85ca70f2146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:59.053279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984fdce-ca6f-4ffd-be56-85ca70f2146a admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:59.053972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984fdce-ca6f-4ffd-be56-85ca70f2146a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:59.055909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984fdce-ca6f-4ffd-be56-85ca70f2146a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:59.061384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:26:59.217498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 991/1995] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:26:59 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:59.222125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:59.233776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c27f2-7d30-4531-a500-c46cd40d3f97 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:59.290075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf825-feb1-45f8-ade9-a73a66acf1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:59.290753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf825-feb1-45f8-ade9-a73a66acf1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:59.291045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf825-feb1-45f8-ade9-a73a66acf1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:59.291413 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf825-feb1-45f8-ade9-a73a66acf1c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:59.292108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caf825-feb1-45f8-ade9-a73a66acf1c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:59.294046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47caf825-feb1-45f8-ade9-a73a66acf1c2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:59.411766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1005/1996] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:26:59 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:26:59.422631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984fdce-ca6f-4ffd-be56-85ca70f2146a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:59.433607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984fdce-ca6f-4ffd-be56-85ca70f2146a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:59.439005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbdbdf-7051-4f5e-aebc-bef66ce314e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:59.439837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbdbdf-7051-4f5e-aebc-bef66ce314e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:59.440195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbdbdf-7051-4f5e-aebc-bef66ce314e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:59.440553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbdbdf-7051-4f5e-aebc-bef66ce314e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:59.441290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbdbdf-7051-4f5e-aebc-bef66ce314e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:59.442960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbdbdf-7051-4f5e-aebc-bef66ce314e7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:59.456328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 992/1997] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:59.460498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf825-feb1-45f8-ade9-a73a66acf1c2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:59.471036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf825-feb1-45f8-ade9-a73a66acf1c2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:59.477003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc411e0-861c-445b-a0c1-b087b665513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:59.477003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc411e0-861c-445b-a0c1-b087b665513d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:59.477003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc411e0-861c-445b-a0c1-b087b665513d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:59.477434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc411e0-861c-445b-a0c1-b087b665513d admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:59.477886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc411e0-861c-445b-a0c1-b087b665513d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:59.479578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc411e0-861c-445b-a0c1-b087b665513d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:59.491042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1006/1998] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:26:59.495480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbdbdf-7051-4f5e-aebc-bef66ce314e7 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:59.506651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbdbdf-7051-4f5e-aebc-bef66ce314e7 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:59.511162 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:59.512074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:59.512363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:59.512639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] PATH_INFO: `/v3/projects/679163b973694df7a7d0fb1184f31f7e/users/0b8f21537fc14c6fabeec0252205e1dc/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:59.526944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=679163b973694df7a7d0fb1184f31f7e, user_id=0b8f21537fc14c6fabeec0252205e1dc, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:59.529708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:59.658155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 993/1999] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:26:59 2022] PUT /identity/v3/projects/679163b973694df7a7d0fb1184f31f7e/users/0b8f21537fc14c6fabeec0252205e1dc/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:26:59.665781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a13b2-3c59-401a-b333-937cc04328b0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:59.666217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a13b2-3c59-401a-b333-937cc04328b0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:59.666541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a13b2-3c59-401a-b333-937cc04328b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:59.692820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-047a13b2-3c59-401a-b333-937cc04328b0 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:26:59.694052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-047a13b2-3c59-401a-b333-937cc04328b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:59.755544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1007/2000] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:26:59.765656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:26:59.767843 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:26:59.832160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6b32b-2e89-4e77-9930-46d3952ffef3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:26:59.836268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:26:59.836917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:26:59.837201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:26:59.837503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:26:59.851512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:26:59.851512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:26:59.870118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 994/2001] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:00.656967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:01.217553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:01.252708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1c77-2630-42a1-ba7c-864e175b8012 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:01.252990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1c77-2630-42a1-ba7c-864e175b8012 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:01.253269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f1c77-2630-42a1-ba7c-864e175b8012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:01.294155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-574f1c77-2630-42a1-ba7c-864e175b8012 None None] MFA Rules not processed for user `0b8f21537fc14c6fabeec0252205e1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:01.295807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-574f1c77-2630-42a1-ba7c-864e175b8012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:01.371245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1008/2002] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:01.388472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:01.388472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:01.450620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bafc9-5c46-4b4e-a411-d108756c92fc service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:01.455109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:01.455857 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:01.456208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:01.456491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:01.465233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:01.466696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:01.477099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 995/2003] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:02.342754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:02.862365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:03.418709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b371-17ea-4df8-b6c9-76ddd4d93628 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:03.418930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b371-17ea-4df8-b6c9-76ddd4d93628 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:03.419214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68b371-17ea-4df8-b6c9-76ddd4d93628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:03.436669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7b68b371-17ea-4df8-b6c9-76ddd4d93628 None None] MFA Rules not processed for user `0b8f21537fc14c6fabeec0252205e1dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:03.438417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7b68b371-17ea-4df8-b6c9-76ddd4d93628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:03.445366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1009/2004] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:03.537464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:03.594901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:04.988213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:05.165254 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:05.744619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:05.746276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:05.766252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a07f6e-b09d-4214-a08e-be2b3052bc95 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:05.770860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:05.778014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:05.778014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:05.778014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] PATH_INFO: `/v3/users/5f9ae0e6a2534bb0b5d5e9327b01b238` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:05.778014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f9ae0e6a2534bb0b5d5e9327b01b238)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:05.778014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:05.987239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f9ae0e6a2534bb0b5d5e9327b01b238'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:05.988011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f9ae0e6a2534bb0b5d5e9327b01b238'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:05.988687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f9ae0e6a2534bb0b5d5e9327b01b238'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:05.996332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f9ae0e6a2534bb0b5d5e9327b01b238'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:06.008547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b1459ba8-5c8c-4e2e-908e-7b304ff0fc82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f9ae0e6a2534bb0b5d5e9327b01b238'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:06.107705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 996/2005] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:05 2022] DELETE /identity/v3/users/5f9ae0e6a2534bb0b5d5e9327b01b238 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:06.409505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68b371-17ea-4df8-b6c9-76ddd4d93628 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:06.423054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68b371-17ea-4df8-b6c9-76ddd4d93628 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:06.485492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:06.486368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:06.486680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:06.486962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] PATH_INFO: `/v3/projects/877612289735450ea6bd527ebe7b81f3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:06.488183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] RBAC: Authorizing `identity:delete_project(project_id=877612289735450ea6bd527ebe7b81f3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:06.490122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:06.575130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877612289735450ea6bd527ebe7b81f3', 'request_id': 'req-06a96ffb-6219-458f-8822-29d09058b18a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:06.584556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877612289735450ea6bd527ebe7b81f3', 'request_id': 'req-06a96ffb-6219-458f-8822-29d09058b18a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:06.645043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:06.678474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877612289735450ea6bd527ebe7b81f3', 'request_id': 'req-06a96ffb-6219-458f-8822-29d09058b18a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:06.690313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] The token cache is being invalidate because project 877612289735450ea6bd527ebe7b81f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:06.695766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:06.695766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06a96ffb-6219-458f-8822-29d09058b18a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:06.695766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1010/2006] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:06 2022] DELETE /identity/v3/projects/877612289735450ea6bd527ebe7b81f3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:06.704273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f4207-86d5-41ea-ac78-ba14d3c9a294 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:06.704821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f4207-86d5-41ea-ac78-ba14d3c9a294 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:06.704821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f4207-86d5-41ea-ac78-ba14d3c9a294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:06.723504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-673f4207-86d5-41ea-ac78-ba14d3c9a294 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:06.724801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-673f4207-86d5-41ea-ac78-ba14d3c9a294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:06.759596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:06.789659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 997/2007] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:06.798544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a274c-8028-44cd-b940-5fe1f821e2cc None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:06.798911 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a274c-8028-44cd-b940-5fe1f821e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:06.799299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a274c-8028-44cd-b940-5fe1f821e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:06.818253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-279a274c-8028-44cd-b940-5fe1f821e2cc None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:06.819496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-279a274c-8028-44cd-b940-5fe1f821e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:06.828381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1011/2008] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:06.832371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f4207-86d5-41ea-ac78-ba14d3c9a294 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:06.841983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f4207-86d5-41ea-ac78-ba14d3c9a294 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:06.846766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:06.847540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:06.847826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:06.848104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:06.848905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:27:06.849222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:06.850877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:06.862680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 998/2009] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:27:06.867220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a274c-8028-44cd-b940-5fe1f821e2cc None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:06.877055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a274c-8028-44cd-b940-5fe1f821e2cc None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:06.881689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30f764-944d-4f7d-81d9-f67ec1b899c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:06.882335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30f764-944d-4f7d-81d9-f67ec1b899c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:06.882617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30f764-944d-4f7d-81d9-f67ec1b899c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:06.882894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30f764-944d-4f7d-81d9-f67ec1b899c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:06.883615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30f764-944d-4f7d-81d9-f67ec1b899c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:06.885532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30f764-944d-4f7d-81d9-f67ec1b899c2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:07.061506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1012/2010] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:06 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:07.063084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:07.073251 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03b5dc-1c48-4d83-9faf-3817e2f2568c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:07.128683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32dda6-b650-4ab3-96a8-17d5f1d544e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:07.129360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32dda6-b650-4ab3-96a8-17d5f1d544e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.129646 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32dda6-b650-4ab3-96a8-17d5f1d544e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.129922 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32dda6-b650-4ab3-96a8-17d5f1d544e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.130625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32dda6-b650-4ab3-96a8-17d5f1d544e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:07.132564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32dda6-b650-4ab3-96a8-17d5f1d544e4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:07.239733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 999/2011] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:07 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:07.243796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30f764-944d-4f7d-81d9-f67ec1b899c2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:07.254358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30f764-944d-4f7d-81d9-f67ec1b899c2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:07.258883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbacd87-5b29-4ca8-829f-cf78f2454328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:07.259579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbacd87-5b29-4ca8-829f-cf78f2454328 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.259867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbacd87-5b29-4ca8-829f-cf78f2454328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.260148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbacd87-5b29-4ca8-829f-cf78f2454328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.260870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbacd87-5b29-4ca8-829f-cf78f2454328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:07.262455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbacd87-5b29-4ca8-829f-cf78f2454328 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:07.272809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1013/2012] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:07.276769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32dda6-b650-4ab3-96a8-17d5f1d544e4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:07.287398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32dda6-b650-4ab3-96a8-17d5f1d544e4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:07.292025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4d86a-1351-4789-8352-5ede33b219c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:07.292682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4d86a-1351-4789-8352-5ede33b219c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.292966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4d86a-1351-4789-8352-5ede33b219c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.293244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4d86a-1351-4789-8352-5ede33b219c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.293975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4d86a-1351-4789-8352-5ede33b219c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:07.295621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4d86a-1351-4789-8352-5ede33b219c7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:07.306520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1000/2013] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:07.310556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbacd87-5b29-4ca8-829f-cf78f2454328 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:07.320414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbacd87-5b29-4ca8-829f-cf78f2454328 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:07.325029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a688cb-9c9f-4157-802b-09aeb402e561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:07.325955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a688cb-9c9f-4157-802b-09aeb402e561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.326244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a688cb-9c9f-4157-802b-09aeb402e561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.326529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a688cb-9c9f-4157-802b-09aeb402e561 admin admin] PATH_INFO: `/v3/projects/86bd8e4c485847849e757e8d493ae4d8/users/2c652f778d7f43d08d492754647ca736/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.340736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a688cb-9c9f-4157-802b-09aeb402e561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bd8e4c485847849e757e8d493ae4d8, user_id=2c652f778d7f43d08d492754647ca736, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:07.343910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a688cb-9c9f-4157-802b-09aeb402e561 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:07.410719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1014/2014] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:07 2022] PUT /identity/v3/projects/86bd8e4c485847849e757e8d493ae4d8/users/2c652f778d7f43d08d492754647ca736/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:07.420683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca10d2a-c719-4ce9-b9f1-f62662acd5ed None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.420984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca10d2a-c719-4ce9-b9f1-f62662acd5ed None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.421307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca10d2a-c719-4ce9-b9f1-f62662acd5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.447933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fca10d2a-c719-4ce9-b9f1-f62662acd5ed None None] MFA Rules not processed for user `2c652f778d7f43d08d492754647ca736`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:07.449170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fca10d2a-c719-4ce9-b9f1-f62662acd5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:07.491382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:07.516123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1001/2015] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:07.520651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a688cb-9c9f-4157-802b-09aeb402e561 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:07.581113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a688cb-9c9f-4157-802b-09aeb402e561 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:07.582281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:07.585831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80bd20-7804-41af-805d-99bfbfcb9345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:07.586487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80bd20-7804-41af-805d-99bfbfcb9345 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.586773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80bd20-7804-41af-805d-99bfbfcb9345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.587052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80bd20-7804-41af-805d-99bfbfcb9345 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.587775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80bd20-7804-41af-805d-99bfbfcb9345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:07.589648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80bd20-7804-41af-805d-99bfbfcb9345 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:07.714161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1015/2016] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:07 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:07.718171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca10d2a-c719-4ce9-b9f1-f62662acd5ed None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:07.729438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca10d2a-c719-4ce9-b9f1-f62662acd5ed None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:07.783616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c584013-9581-4225-9781-39d2749f828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:07.784282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c584013-9581-4225-9781-39d2749f828b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.784570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c584013-9581-4225-9781-39d2749f828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.784846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c584013-9581-4225-9781-39d2749f828b admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.785547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c584013-9581-4225-9781-39d2749f828b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:07.787534 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c584013-9581-4225-9781-39d2749f828b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:07.876856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1002/2017] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:07 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:27:07.878417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80bd20-7804-41af-805d-99bfbfcb9345 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:07.888912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80bd20-7804-41af-805d-99bfbfcb9345 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:07.894113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674f025-db5d-4d3b-98b3-4f705b6ca520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:07.894785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674f025-db5d-4d3b-98b3-4f705b6ca520 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.895074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674f025-db5d-4d3b-98b3-4f705b6ca520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.895375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674f025-db5d-4d3b-98b3-4f705b6ca520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.896129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674f025-db5d-4d3b-98b3-4f705b6ca520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:07.897710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674f025-db5d-4d3b-98b3-4f705b6ca520 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:07.908471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1016/2018] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:27:07.912840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c584013-9581-4225-9781-39d2749f828b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:07.922993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c584013-9581-4225-9781-39d2749f828b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:07.927721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d459f1-bf36-4dff-854d-cd9efa75bc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:07.928599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d459f1-bf36-4dff-854d-cd9efa75bc5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:07.928878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d459f1-bf36-4dff-854d-cd9efa75bc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:07.929152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d459f1-bf36-4dff-854d-cd9efa75bc5a admin admin] PATH_INFO: `/v3/projects/b4a1ef5ddb6e4af6a2d1d7716b90896c/users/03a17b9d181542029162a835b308270c/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:07.947759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d459f1-bf36-4dff-854d-cd9efa75bc5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a1ef5ddb6e4af6a2d1d7716b90896c, user_id=03a17b9d181542029162a835b308270c, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:07.952432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d459f1-bf36-4dff-854d-cd9efa75bc5a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:08.056594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1003/2019] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:07 2022] PUT /identity/v3/projects/b4a1ef5ddb6e4af6a2d1d7716b90896c/users/03a17b9d181542029162a835b308270c/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:08.060657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674f025-db5d-4d3b-98b3-4f705b6ca520 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:08.070505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:08.073110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674f025-db5d-4d3b-98b3-4f705b6ca520 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:08.132107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308151b0-e5f7-495d-8d44-e964dd511f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:08.132776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308151b0-e5f7-495d-8d44-e964dd511f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:08.133060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308151b0-e5f7-495d-8d44-e964dd511f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:08.133337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308151b0-e5f7-495d-8d44-e964dd511f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:08.134056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308151b0-e5f7-495d-8d44-e964dd511f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:08.135673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308151b0-e5f7-495d-8d44-e964dd511f13 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:08.136522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:08.145230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1017/2020] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:08.149500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d459f1-bf36-4dff-854d-cd9efa75bc5a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:08.159742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d459f1-bf36-4dff-854d-cd9efa75bc5a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:08.164288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:08.165160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:08.165437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:08.165710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] PATH_INFO: `/v3/projects/b4a1ef5ddb6e4af6a2d1d7716b90896c/users/03a17b9d181542029162a835b308270c/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:08.167680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a1ef5ddb6e4af6a2d1d7716b90896c, user_id=03a17b9d181542029162a835b308270c, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:08.170289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:08.182018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1004/2021] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:08 2022] PUT /identity/v3/projects/b4a1ef5ddb6e4af6a2d1d7716b90896c/users/03a17b9d181542029162a835b308270c/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:08.195695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aebe6-924a-4f8d-93f6-e127d6025e19 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:08.196255 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aebe6-924a-4f8d-93f6-e127d6025e19 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:08.196749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aebe6-924a-4f8d-93f6-e127d6025e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:08.227725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9e1aebe6-924a-4f8d-93f6-e127d6025e19 None None] MFA Rules not processed for user `03a17b9d181542029162a835b308270c`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:08.228968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9e1aebe6-924a-4f8d-93f6-e127d6025e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:08.300474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1018/2022] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:08.314371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:08.316157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:08.371510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c0da-e77c-4b19-9a0b-59dcba1e1309 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:08.375728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eca293-dc2c-4f8b-932d-d0cb16824d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:08.376358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca293-dc2c-4f8b-932d-d0cb16824d56 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:08.376638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca293-dc2c-4f8b-932d-d0cb16824d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:08.376910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca293-dc2c-4f8b-932d-d0cb16824d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:08.385732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eca293-dc2c-4f8b-932d-d0cb16824d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:08.387193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eca293-dc2c-4f8b-932d-d0cb16824d56 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:08.396532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1005/2023] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:08.428183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:08.524838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:09.021062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:09.232284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:09.660663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:09.857346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aebe6-924a-4f8d-93f6-e127d6025e19 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:09.867001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aebe6-924a-4f8d-93f6-e127d6025e19 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:09.871846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a17b9d181542029162a835b308270c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4a1ef5ddb6e4af6a2d1d7716b90896c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:09.872581 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:09.872870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:09.873216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:09.874006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:27:09.874326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:09.876605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:09.891850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1019/2024] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:27:09.893058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eca293-dc2c-4f8b-932d-d0cb16824d56 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:09.902442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eca293-dc2c-4f8b-932d-d0cb16824d56 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:09.909813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a17b9d181542029162a835b308270c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4a1ef5ddb6e4af6a2d1d7716b90896c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:09.910570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:09.910951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:09.911467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:09.912379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:09.914480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:10.034074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:10.086775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1006/2025] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:09 2022] POST /identity/v3/projects => generated 355 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:10.087932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:10.099410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:10.104980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:10.105601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd368d-60a7-4e07-a5bd-46630f3fd5a0 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:10.180013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a17b9d181542029162a835b308270c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4a1ef5ddb6e4af6a2d1d7716b90896c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:10.181757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:10.182126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:10.182473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:10.183364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:27:10.183804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:10.186981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:10.203775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1020/2026] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:27:10.203955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c957b-a30f-49ad-862a-aaca4e040be9 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:10.210556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:10.211053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:10.211826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:10.212145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] PATH_INFO: `/v3/users/8146247ce06e4730b9d3169ec263f786` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:10.221444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8146247ce06e4730b9d3169ec263f786)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:10.224311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:10.244206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:10.255458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf80f-14fd-444a-86d6-29213f4bf089 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:10.260431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03a17b9d181542029162a835b308270c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4a1ef5ddb6e4af6a2d1d7716b90896c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:10.261310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:10.261603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:10.261887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] PATH_INFO: `/v3/projects/29c7daca1a504217bd0bbef3b27a0e6c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:10.263132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Authorizing `identity:delete_project(project_id=29c7daca1a504217bd0bbef3b27a0e6c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:10.266416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:10.599460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:10.614830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8146247ce06e4730b9d3169ec263f786'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.615165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c7daca1a504217bd0bbef3b27a0e6c', 'request_id': 'req-2ea167d7-4775-4361-9907-e4a1056d1c19'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.615700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8146247ce06e4730b9d3169ec263f786'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.616363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8146247ce06e4730b9d3169ec263f786'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.620699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c7daca1a504217bd0bbef3b27a0e6c', 'request_id': 'req-2ea167d7-4775-4361-9907-e4a1056d1c19'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.623967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8146247ce06e4730b9d3169ec263f786'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.636578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8146247ce06e4730b9d3169ec263f786'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.759671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c7daca1a504217bd0bbef3b27a0e6c', 'request_id': 'req-2ea167d7-4775-4361-9907-e4a1056d1c19'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.776800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] The token cache is being invalidate because project 29c7daca1a504217bd0bbef3b27a0e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:10.777476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.778788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2ea167d7-4775-4361-9907-e4a1056d1c19 tempest-VolumeQuotaClassesTest-1648294512 tempest-VolumeQuotaClassesTest-1648294512-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:10.782197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1021/2027] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:10 2022] DELETE /identity/v3/projects/29c7daca1a504217bd0bbef3b27a0e6c => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:10.823031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:10.872733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1007/2028] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:10 2022] DELETE /identity/v3/users/8146247ce06e4730b9d3169ec263f786 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:10.877825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281660-22b1-429f-bcc0-a22507fb131a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:10.878356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281660-22b1-429f-bcc0-a22507fb131a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:10.878447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281660-22b1-429f-bcc0-a22507fb131a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:10.897280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-53281660-22b1-429f-bcc0-a22507fb131a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:10.898542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-53281660-22b1-429f-bcc0-a22507fb131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:10.961811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1022/2029] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:10.971635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:10.973055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:11.031766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267652b2-26b8-47f6-940b-22178c1362aa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:11.035405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:11.036050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:11.036330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:11.036605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:11.046396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:11.047989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:11.061261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1008/2030] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:11.677725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53281660-22b1-429f-bcc0-a22507fb131a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:11.679294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-53281660-22b1-429f-bcc0-a22507fb131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:11.693973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:11.696541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:11.706278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53281660-22b1-429f-bcc0-a22507fb131a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:11.714619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e295683-688b-40f3-93c1-39dc6f58a143 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:11.716776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:11.717691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:11.717968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:11.718422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] PATH_INFO: `/v3/users/2c652f778d7f43d08d492754647ca736` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:11.720008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c652f778d7f43d08d492754647ca736)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:11.721189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:11.722425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:11.722883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:11.723417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:11.724284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] PATH_INFO: `/v3/projects/e9734df9b60e4e239c2df914a70c5c7f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:11.726548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9734df9b60e4e239c2df914a70c5c7f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:11.729931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:11.760577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:11.952438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c652f778d7f43d08d492754647ca736'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:11.953202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c652f778d7f43d08d492754647ca736'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:11.953879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c652f778d7f43d08d492754647ca736'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:11.957265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9734df9b60e4e239c2df914a70c5c7f', 'request_id': 'req-c4c719e4-5766-468f-9f3d-347d30bb0102'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:11.961329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c652f778d7f43d08d492754647ca736'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:11.963996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9734df9b60e4e239c2df914a70c5c7f', 'request_id': 'req-c4c719e4-5766-468f-9f3d-347d30bb0102'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:11.973885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3603588-bc77-4e89-b1d5-3cb885b09336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c652f778d7f43d08d492754647ca736'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:12.095279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9734df9b60e4e239c2df914a70c5c7f', 'request_id': 'req-c4c719e4-5766-468f-9f3d-347d30bb0102'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:12.107156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] The token cache is being invalidate because project e9734df9b60e4e239c2df914a70c5c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:12.107479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:12.108167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c4c719e4-5766-468f-9f3d-347d30bb0102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:12.109727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1009/2031] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:11 2022] DELETE /identity/v3/projects/e9734df9b60e4e239c2df914a70c5c7f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:27:12.115106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21ebb-7042-477f-b9ef-3261a6232f1a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.115462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21ebb-7042-477f-b9ef-3261a6232f1a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.115744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21ebb-7042-477f-b9ef-3261a6232f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.145696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-74e21ebb-7042-477f-b9ef-3261a6232f1a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:12.145696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-74e21ebb-7042-477f-b9ef-3261a6232f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:12.196681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1010/2032] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:12.205370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6695c0-0b01-486f-83ee-e7a73ef21c47 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.205668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6695c0-0b01-486f-83ee-e7a73ef21c47 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.205947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6695c0-0b01-486f-83ee-e7a73ef21c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.224536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7c6695c0-0b01-486f-83ee-e7a73ef21c47 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:12.225884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7c6695c0-0b01-486f-83ee-e7a73ef21c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:12.240277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1011/2033] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:27:12.242054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1023/2034] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:11 2022] DELETE /identity/v3/users/2c652f778d7f43d08d492754647ca736 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:12.242371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6695c0-0b01-486f-83ee-e7a73ef21c47 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:12.248289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0eeff1-65d4-48d7-bacd-54f1337acbcb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.248852 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0eeff1-65d4-48d7-bacd-54f1337acbcb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.249412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0eeff1-65d4-48d7-bacd-54f1337acbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.264872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6695c0-0b01-486f-83ee-e7a73ef21c47 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:12.277989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-af0eeff1-65d4-48d7-bacd-54f1337acbcb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:12.279361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-af0eeff1-65d4-48d7-bacd-54f1337acbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:12.325228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:12.325907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.326194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.326476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.327216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:27:12.327598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:12.329273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:12.341406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1012/2035] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:27:12.347456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1024/2036] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:12.348106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:12.359344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326f692-a1d3-461a-b141-2ba81b32fae1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:12.364063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0eeff1-65d4-48d7-bacd-54f1337acbcb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:12.365545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-af0eeff1-65d4-48d7-bacd-54f1337acbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:12.366525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f74b05-8476-44b2-9bb6-b6cd6e64968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:12.367195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74b05-8476-44b2-9bb6-b6cd6e64968c admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.367532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74b05-8476-44b2-9bb6-b6cd6e64968c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.367813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74b05-8476-44b2-9bb6-b6cd6e64968c admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.368531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f74b05-8476-44b2-9bb6-b6cd6e64968c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:12.370544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f74b05-8476-44b2-9bb6-b6cd6e64968c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:12.425311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0eeff1-65d4-48d7-bacd-54f1337acbcb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:12.429536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d28f38-0bc7-4950-9bb0-bc6c155e7a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:12.430186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d28f38-0bc7-4950-9bb0-bc6c155e7a3c service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.430471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d28f38-0bc7-4950-9bb0-bc6c155e7a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.430751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d28f38-0bc7-4950-9bb0-bc6c155e7a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.440476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d28f38-0bc7-4950-9bb0-bc6c155e7a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:12.441904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d28f38-0bc7-4950-9bb0-bc6c155e7a3c service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:12.460160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1025/2037] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:27:12.504530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1013/2038] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:12 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:12.508968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d28f38-0bc7-4950-9bb0-bc6c155e7a3c service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:12.518678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d28f38-0bc7-4950-9bb0-bc6c155e7a3c service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:12.575896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29174b-4316-417c-9e87-53656e5d0554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:12.576599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29174b-4316-417c-9e87-53656e5d0554 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.576883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29174b-4316-417c-9e87-53656e5d0554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.577193 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29174b-4316-417c-9e87-53656e5d0554 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.577931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29174b-4316-417c-9e87-53656e5d0554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:12.580486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b29174b-4316-417c-9e87-53656e5d0554 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:12.589373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:12.653607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1026/2039] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:12 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:12.656768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f74b05-8476-44b2-9bb6-b6cd6e64968c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:12.671530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f74b05-8476-44b2-9bb6-b6cd6e64968c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:12.677623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e9750-fad9-4a6c-a73f-9be337afc5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:12.678478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e9750-fad9-4a6c-a73f-9be337afc5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.678771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e9750-fad9-4a6c-a73f-9be337afc5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.679054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e9750-fad9-4a6c-a73f-9be337afc5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.679848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e9750-fad9-4a6c-a73f-9be337afc5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:12.681474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e9750-fad9-4a6c-a73f-9be337afc5a7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:12.695110 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1014/2040] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:27:12.696327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29174b-4316-417c-9e87-53656e5d0554 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:12.707030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29174b-4316-417c-9e87-53656e5d0554 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:12.712122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7c9fa-8a9f-4ed2-8a90-c438d4f101a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:12.712809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c9fa-8a9f-4ed2-8a90-c438d4f101a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.713097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c9fa-8a9f-4ed2-8a90-c438d4f101a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.713378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c9fa-8a9f-4ed2-8a90-c438d4f101a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.714113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7c9fa-8a9f-4ed2-8a90-c438d4f101a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:12.715767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7c9fa-8a9f-4ed2-8a90-c438d4f101a4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:12.731295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1027/2041] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:12.733666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e9750-fad9-4a6c-a73f-9be337afc5a7 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:12.744602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e9750-fad9-4a6c-a73f-9be337afc5a7 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:12.749860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:12.750751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.751066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.751377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] PATH_INFO: `/v3/projects/1354b2d4544c487489ab820e9793e60a/users/f1984ac56bc84ae29499a95f9b444bc0/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.769051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1354b2d4544c487489ab820e9793e60a, user_id=f1984ac56bc84ae29499a95f9b444bc0, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:12.772201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:12.856768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1015/2042] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:12 2022] PUT /identity/v3/projects/1354b2d4544c487489ab820e9793e60a/users/f1984ac56bc84ae29499a95f9b444bc0/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:27:12.873484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16186-4ded-45df-9319-277db59025b3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.873778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16186-4ded-45df-9319-277db59025b3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.874093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16186-4ded-45df-9319-277db59025b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.895416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:12.898315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:12.912872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e8c16186-4ded-45df-9319-277db59025b3 None None] MFA Rules not processed for user `f1984ac56bc84ae29499a95f9b444bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:12.914306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e8c16186-4ded-45df-9319-277db59025b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:12.968036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a718d95-e74b-42e3-9695-14eab98a37fe admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:12.973270 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:12.974148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:12.974424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:12.974701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] PATH_INFO: `/v3/projects/86bd8e4c485847849e757e8d493ae4d8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:12.975908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bd8e4c485847849e757e8d493ae4d8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:12.977837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:12.986822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1028/2043] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:12.999212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16186-4ded-45df-9319-277db59025b3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:13.000676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e8c16186-4ded-45df-9319-277db59025b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:13.058024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16186-4ded-45df-9319-277db59025b3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:13.062260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b63fa-2814-43c0-9543-575d0684c134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:13.062931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b63fa-2814-43c0-9543-575d0684c134 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:13.063216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b63fa-2814-43c0-9543-575d0684c134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:13.064017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b63fa-2814-43c0-9543-575d0684c134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:13.072758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b63fa-2814-43c0-9543-575d0684c134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:13.074218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b63fa-2814-43c0-9543-575d0684c134 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:13.083786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1029/2044] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:27:13.107739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bd8e4c485847849e757e8d493ae4d8', 'request_id': 'req-d6f293aa-7b48-4b80-b939-39efc422db22'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.113366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bd8e4c485847849e757e8d493ae4d8', 'request_id': 'req-d6f293aa-7b48-4b80-b939-39efc422db22'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.204719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bd8e4c485847849e757e8d493ae4d8', 'request_id': 'req-d6f293aa-7b48-4b80-b939-39efc422db22'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.217673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] The token cache is being invalidate because project 86bd8e4c485847849e757e8d493ae4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:13.217984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.218761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.220540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1016/2045] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:12 2022] DELETE /identity/v3/projects/86bd8e4c485847849e757e8d493ae4d8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:13.224611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b63fa-2814-43c0-9543-575d0684c134 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:13.235604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b63fa-2814-43c0-9543-575d0684c134 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:13.292692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:13.293586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:13.293862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:13.294255 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] PATH_INFO: `/v3/users/03a17b9d181542029162a835b308270c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:13.295538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a17b9d181542029162a835b308270c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:13.297500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:13.519137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a17b9d181542029162a835b308270c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.519812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a17b9d181542029162a835b308270c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.520424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a17b9d181542029162a835b308270c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.527346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a17b9d181542029162a835b308270c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.538937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f5b63120-2325-4385-a712-bc5e0af70d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a17b9d181542029162a835b308270c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:13.616465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1030/2046] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:13 2022] DELETE /identity/v3/users/03a17b9d181542029162a835b308270c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:14.044711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:14.284813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:14.349000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:14.350432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:14.364217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f293aa-7b48-4b80-b939-39efc422db22 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:14.421131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:14.422243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:14.422524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:14.422820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] PATH_INFO: `/v3/projects/b4a1ef5ddb6e4af6a2d1d7716b90896c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:14.424030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a1ef5ddb6e4af6a2d1d7716b90896c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:14.426366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:14.565095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a1ef5ddb6e4af6a2d1d7716b90896c', 'request_id': 'req-89b8b77f-2734-449b-8caa-1648c2477cc0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:14.570481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a1ef5ddb6e4af6a2d1d7716b90896c', 'request_id': 'req-89b8b77f-2734-449b-8caa-1648c2477cc0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:14.656165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a1ef5ddb6e4af6a2d1d7716b90896c', 'request_id': 'req-89b8b77f-2734-449b-8caa-1648c2477cc0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:14.668460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] The token cache is being invalidate because project b4a1ef5ddb6e4af6a2d1d7716b90896c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:14.668751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:14.669413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-89b8b77f-2734-449b-8caa-1648c2477cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:14.671107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1017/2047] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:14 2022] DELETE /identity/v3/projects/b4a1ef5ddb6e4af6a2d1d7716b90896c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:14.676713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e754d49-5fce-428b-a95f-ae1fd4a08c0c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:14.677030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e754d49-5fce-428b-a95f-ae1fd4a08c0c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:14.677452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e754d49-5fce-428b-a95f-ae1fd4a08c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:14.695718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9e754d49-5fce-428b-a95f-ae1fd4a08c0c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:14.696963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9e754d49-5fce-428b-a95f-ae1fd4a08c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:14.757069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1031/2048] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:14.765658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a771c-2f5a-48e8-beb9-721713bbcc87 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:14.765948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a771c-2f5a-48e8-beb9-721713bbcc87 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:14.766226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a771c-2f5a-48e8-beb9-721713bbcc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:14.784760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-de5a771c-2f5a-48e8-beb9-721713bbcc87 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:14.785946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-de5a771c-2f5a-48e8-beb9-721713bbcc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:14.794103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1018/2049] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:14.798799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e754d49-5fce-428b-a95f-ae1fd4a08c0c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:14.808509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e754d49-5fce-428b-a95f-ae1fd4a08c0c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:14.813668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:14.814343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:14.814626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:14.819928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:14.819928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:27:14.819928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:14.819928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:14.829619 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1032/2050] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:27:14.834014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a771c-2f5a-48e8-beb9-721713bbcc87 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:14.843632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a771c-2f5a-48e8-beb9-721713bbcc87 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:14.848853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe57ec0-156a-4a34-9cb6-c6869c8d088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:14.849501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57ec0-156a-4a34-9cb6-c6869c8d088a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:14.849786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57ec0-156a-4a34-9cb6-c6869c8d088a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:14.851354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57ec0-156a-4a34-9cb6-c6869c8d088a admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:14.851354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe57ec0-156a-4a34-9cb6-c6869c8d088a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:14.852658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe57ec0-156a-4a34-9cb6-c6869c8d088a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:14.995229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1019/2051] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:14 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:14.999453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:15.009579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a326d36-4919-4842-8ac7-0c58a8fa6b3a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:15.066582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496ed7-26e9-497e-b0cf-68706e7d40a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:15.067244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496ed7-26e9-497e-b0cf-68706e7d40a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:15.067559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496ed7-26e9-497e-b0cf-68706e7d40a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:15.067845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496ed7-26e9-497e-b0cf-68706e7d40a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:15.068552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e496ed7-26e9-497e-b0cf-68706e7d40a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:15.070576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e496ed7-26e9-497e-b0cf-68706e7d40a6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:15.217050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1033/2052] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:14 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:15.221329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe57ec0-156a-4a34-9cb6-c6869c8d088a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:15.231713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe57ec0-156a-4a34-9cb6-c6869c8d088a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:15.239956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fa1bb-0ec9-4e8c-8bcd-e4de318da997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:15.240794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fa1bb-0ec9-4e8c-8bcd-e4de318da997 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:15.241085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fa1bb-0ec9-4e8c-8bcd-e4de318da997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:15.241369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fa1bb-0ec9-4e8c-8bcd-e4de318da997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:15.242105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fa1bb-0ec9-4e8c-8bcd-e4de318da997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:15.243807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fa1bb-0ec9-4e8c-8bcd-e4de318da997 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:15.255442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1020/2053] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:15.259556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496ed7-26e9-497e-b0cf-68706e7d40a6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:15.270919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496ed7-26e9-497e-b0cf-68706e7d40a6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:15.275827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc97464-98f0-461e-b0e3-360654731558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:15.276444 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97464-98f0-461e-b0e3-360654731558 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:15.276731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97464-98f0-461e-b0e3-360654731558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:15.277010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97464-98f0-461e-b0e3-360654731558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:15.277896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc97464-98f0-461e-b0e3-360654731558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:15.279522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc97464-98f0-461e-b0e3-360654731558 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:15.294690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1034/2054] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:15.295625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fa1bb-0ec9-4e8c-8bcd-e4de318da997 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:15.308095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fa1bb-0ec9-4e8c-8bcd-e4de318da997 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:15.317259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462939a1-fd57-4d82-823c-75f4dac719ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:15.318613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462939a1-fd57-4d82-823c-75f4dac719ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:15.319058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462939a1-fd57-4d82-823c-75f4dac719ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:15.319491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462939a1-fd57-4d82-823c-75f4dac719ea admin admin] PATH_INFO: `/v3/projects/c583ff5d62bd4bb0b090dbaae715274d/users/54a5c49ebb0846ea8c3092f62a711892/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:15.336205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462939a1-fd57-4d82-823c-75f4dac719ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c583ff5d62bd4bb0b090dbaae715274d, user_id=54a5c49ebb0846ea8c3092f62a711892, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:15.338944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462939a1-fd57-4d82-823c-75f4dac719ea admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:15.432901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1021/2055] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:15 2022] PUT /identity/v3/projects/c583ff5d62bd4bb0b090dbaae715274d/users/54a5c49ebb0846ea8c3092f62a711892/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:15.443734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a12f3-8e14-480a-8011-ee8e2808b838 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:15.444023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a12f3-8e14-480a-8011-ee8e2808b838 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:15.444301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a12f3-8e14-480a-8011-ee8e2808b838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:15.472378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ad8a12f3-8e14-480a-8011-ee8e2808b838 None None] MFA Rules not processed for user `54a5c49ebb0846ea8c3092f62a711892`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:15.473630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ad8a12f3-8e14-480a-8011-ee8e2808b838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:15.542249 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1035/2056] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:15.553077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462939a1-fd57-4d82-823c-75f4dac719ea admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:15.622765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462939a1-fd57-4d82-823c-75f4dac719ea admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:15.635626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a33c6-91e2-4ecb-9c18-ff083a8205e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:15.636483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a33c6-91e2-4ecb-9c18-ff083a8205e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:15.636815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a33c6-91e2-4ecb-9c18-ff083a8205e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:15.637657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a33c6-91e2-4ecb-9c18-ff083a8205e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:15.637898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a33c6-91e2-4ecb-9c18-ff083a8205e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:15.640088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a33c6-91e2-4ecb-9c18-ff083a8205e0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:15.780092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1022/2057] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:15 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:15.783471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a12f3-8e14-480a-8011-ee8e2808b838 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:15.796312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a12f3-8e14-480a-8011-ee8e2808b838 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:15.853256 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda7472-e8fe-47e4-a878-dcb135c1173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:15.853928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda7472-e8fe-47e4-a878-dcb135c1173f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:15.854212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda7472-e8fe-47e4-a878-dcb135c1173f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:15.854488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda7472-e8fe-47e4-a878-dcb135c1173f admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:15.855185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda7472-e8fe-47e4-a878-dcb135c1173f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:15.857151 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda7472-e8fe-47e4-a878-dcb135c1173f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.032598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1036/2058] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:15 2022] POST /identity/v3/users => generated 294 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:27:16.036475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a33c6-91e2-4ecb-9c18-ff083a8205e0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.046528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a33c6-91e2-4ecb-9c18-ff083a8205e0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.059381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f226-4d5e-4da0-9492-ae8a404e6fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.060447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f226-4d5e-4da0-9492-ae8a404e6fef admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.060758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f226-4d5e-4da0-9492-ae8a404e6fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.061037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230f226-4d5e-4da0-9492-ae8a404e6fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.061772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230f226-4d5e-4da0-9492-ae8a404e6fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.063996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230f226-4d5e-4da0-9492-ae8a404e6fef admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.075039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1023/2059] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:16.082779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda7472-e8fe-47e4-a878-dcb135c1173f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.092792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda7472-e8fe-47e4-a878-dcb135c1173f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.097769 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bee065-87f8-46be-9dd3-42a6c69bf86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.098442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bee065-87f8-46be-9dd3-42a6c69bf86b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.098732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bee065-87f8-46be-9dd3-42a6c69bf86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.099456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bee065-87f8-46be-9dd3-42a6c69bf86b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.099881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bee065-87f8-46be-9dd3-42a6c69bf86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.101491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bee065-87f8-46be-9dd3-42a6c69bf86b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.114672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1037/2060] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:16.115931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f226-4d5e-4da0-9492-ae8a404e6fef admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.125796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230f226-4d5e-4da0-9492-ae8a404e6fef admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.130924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4f59f-3dd2-4b7f-96e9-ad83bdb3734a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.132433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4f59f-3dd2-4b7f-96e9-ad83bdb3734a admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.132433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4f59f-3dd2-4b7f-96e9-ad83bdb3734a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.132433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d4f59f-3dd2-4b7f-96e9-ad83bdb3734a admin admin] PATH_INFO: `/v3/projects/ecd3bf6acc71496d929828817c7f6152/users/c97793c0f6d34022bbfdbe504b049c09/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.147537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4f59f-3dd2-4b7f-96e9-ad83bdb3734a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd3bf6acc71496d929828817c7f6152, user_id=c97793c0f6d34022bbfdbe504b049c09, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.150177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d4f59f-3dd2-4b7f-96e9-ad83bdb3734a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.216839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1024/2061] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:16 2022] PUT /identity/v3/projects/ecd3bf6acc71496d929828817c7f6152/users/c97793c0f6d34022bbfdbe504b049c09/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:16.228119 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac244-c588-475a-a363-8c36f926c2d2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.228418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac244-c588-475a-a363-8c36f926c2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.228732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac244-c588-475a-a363-8c36f926c2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.257253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-012ac244-c588-475a-a363-8c36f926c2d2 None None] MFA Rules not processed for user `c97793c0f6d34022bbfdbe504b049c09`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:16.258596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-012ac244-c588-475a-a363-8c36f926c2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:16.333888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1038/2062] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:16.338519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4f59f-3dd2-4b7f-96e9-ad83bdb3734a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.399343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d4f59f-3dd2-4b7f-96e9-ad83bdb3734a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.404058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d661c-45a5-4b81-8b49-cba75b3657f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.404705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d661c-45a5-4b81-8b49-cba75b3657f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.404987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d661c-45a5-4b81-8b49-cba75b3657f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.405262 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d661c-45a5-4b81-8b49-cba75b3657f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.405959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d661c-45a5-4b81-8b49-cba75b3657f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.407930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d661c-45a5-4b81-8b49-cba75b3657f2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.496936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1025/2063] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:16 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:16.500918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ac244-c588-475a-a363-8c36f926c2d2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.512063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ac244-c588-475a-a363-8c36f926c2d2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.566363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74343da-59b8-497e-9433-0d003dd88c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.567054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74343da-59b8-497e-9433-0d003dd88c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.568861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74343da-59b8-497e-9433-0d003dd88c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.569168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74343da-59b8-497e-9433-0d003dd88c78 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.569889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74343da-59b8-497e-9433-0d003dd88c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.571865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74343da-59b8-497e-9433-0d003dd88c78 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.661502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1039/2064] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:16 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:27:16.665426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d661c-45a5-4b81-8b49-cba75b3657f2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.676317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d661c-45a5-4b81-8b49-cba75b3657f2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.680856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5f8ef-e6ca-4bb3-a52f-04469a815a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.681504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5f8ef-e6ca-4bb3-a52f-04469a815a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.681788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5f8ef-e6ca-4bb3-a52f-04469a815a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.682068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5f8ef-e6ca-4bb3-a52f-04469a815a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.682792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5f8ef-e6ca-4bb3-a52f-04469a815a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.684432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5f8ef-e6ca-4bb3-a52f-04469a815a80 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.696209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1026/2065] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:16.700784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74343da-59b8-497e-9433-0d003dd88c78 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.711149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74343da-59b8-497e-9433-0d003dd88c78 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.714261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:16.717472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc2d36-d903-4367-a8df-14d658b1a980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.718353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc2d36-d903-4367-a8df-14d658b1a980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.718678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc2d36-d903-4367-a8df-14d658b1a980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.720312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc2d36-d903-4367-a8df-14d658b1a980 admin admin] PATH_INFO: `/v3/projects/04c265572ac34a34a020d599d4b87f63/users/ee94cd918269404e83657abf4467c8a3/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.734721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc2d36-d903-4367-a8df-14d658b1a980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c265572ac34a34a020d599d4b87f63, user_id=ee94cd918269404e83657abf4467c8a3, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.738041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc2d36-d903-4367-a8df-14d658b1a980 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.782878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:16.816464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1040/2066] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:16 2022] PUT /identity/v3/projects/04c265572ac34a34a020d599d4b87f63/users/ee94cd918269404e83657abf4467c8a3/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:16.820432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5f8ef-e6ca-4bb3-a52f-04469a815a80 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.830456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5f8ef-e6ca-4bb3-a52f-04469a815a80 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.886250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4200d-be91-41af-b658-d35c937dfe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.886974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4200d-be91-41af-b658-d35c937dfe49 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.887271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4200d-be91-41af-b658-d35c937dfe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.887561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4200d-be91-41af-b658-d35c937dfe49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.888292 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4200d-be91-41af-b658-d35c937dfe49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.889909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4200d-be91-41af-b658-d35c937dfe49 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.899836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1027/2067] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:16.904183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc2d36-d903-4367-a8df-14d658b1a980 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:16.915478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc2d36-d903-4367-a8df-14d658b1a980 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:16.919946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:16.920837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.921125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.921405 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] PATH_INFO: `/v3/projects/04c265572ac34a34a020d599d4b87f63/users/ee94cd918269404e83657abf4467c8a3/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.924011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c265572ac34a34a020d599d4b87f63, user_id=ee94cd918269404e83657abf4467c8a3, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:16.926951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:16.937160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1041/2068] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:16 2022] PUT /identity/v3/projects/04c265572ac34a34a020d599d4b87f63/users/ee94cd918269404e83657abf4467c8a3/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:16.947118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff924de1-780e-4753-bf67-f2a0a2a02be5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:16.947443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff924de1-780e-4753-bf67-f2a0a2a02be5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:16.947725 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff924de1-780e-4753-bf67-f2a0a2a02be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:16.976315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ff924de1-780e-4753-bf67-f2a0a2a02be5 None None] MFA Rules not processed for user `ee94cd918269404e83657abf4467c8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:16.977584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ff924de1-780e-4753-bf67-f2a0a2a02be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:17.042712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1028/2069] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:17.054230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:17.056038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:17.114593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bbe18-65ac-4db2-842b-f7ddf0e3369a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:17.118368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0850e-a581-4f0d-b80c-a3c0183d4e97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:17.119015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0850e-a581-4f0d-b80c-a3c0183d4e97 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:17.119345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0850e-a581-4f0d-b80c-a3c0183d4e97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:17.119630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0850e-a581-4f0d-b80c-a3c0183d4e97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:17.130197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0850e-a581-4f0d-b80c-a3c0183d4e97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:17.131702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0850e-a581-4f0d-b80c-a3c0183d4e97 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:17.142121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1042/2070] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:18.279581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:18.434855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff924de1-780e-4753-bf67-f2a0a2a02be5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:18.436265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ff924de1-780e-4753-bf67-f2a0a2a02be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:18.448453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff924de1-780e-4753-bf67-f2a0a2a02be5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:18.451959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be0b2e-843e-46cf-8c06-046ff19da108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:18.452591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be0b2e-843e-46cf-8c06-046ff19da108 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:18.452887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be0b2e-843e-46cf-8c06-046ff19da108 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:18.453154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be0b2e-843e-46cf-8c06-046ff19da108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:18.507901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be0b2e-843e-46cf-8c06-046ff19da108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:18.509338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be0b2e-843e-46cf-8c06-046ff19da108 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:18.518394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1029/2071] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:18.541835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:19.199127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:19.227922 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:19.651676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:19.696658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:20.142582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:20.382167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:20.582302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:20.719022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:20.747652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:20.970447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:21.142265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:27.216639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0850e-a581-4f0d-b80c-a3c0183d4e97 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:27.226465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0850e-a581-4f0d-b80c-a3c0183d4e97 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:27.230171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31a0be-897f-4a65-aa81-117fdad6f955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:27.230827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31a0be-897f-4a65-aa81-117fdad6f955 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:27.231114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31a0be-897f-4a65-aa81-117fdad6f955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:27.231458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31a0be-897f-4a65-aa81-117fdad6f955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:27.289267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31a0be-897f-4a65-aa81-117fdad6f955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:27.296206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31a0be-897f-4a65-aa81-117fdad6f955 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:27.305754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1043/2072] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:30.850238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=29 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:32.716755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be0b2e-843e-46cf-8c06-046ff19da108 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:32.783227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be0b2e-843e-46cf-8c06-046ff19da108 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:32.787786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:32.788645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:32.788927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:32.789203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] PATH_INFO: `/v3/users/54a5c49ebb0846ea8c3092f62a711892` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:32.790371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a5c49ebb0846ea8c3092f62a711892)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:32.792333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:32.967527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a5c49ebb0846ea8c3092f62a711892'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:32.968244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a5c49ebb0846ea8c3092f62a711892'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:32.968861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a5c49ebb0846ea8c3092f62a711892'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:32.976081 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a5c49ebb0846ea8c3092f62a711892'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:32.988246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a5c49ebb0846ea8c3092f62a711892'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:33.088387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1030/2073] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:32 2022] DELETE /identity/v3/users/54a5c49ebb0846ea8c3092f62a711892 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:33.093475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03955154-50ac-46dc-9ab4-1e8086ce64cb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:33.094004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03955154-50ac-46dc-9ab4-1e8086ce64cb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:33.094121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03955154-50ac-46dc-9ab4-1e8086ce64cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:33.111407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-03955154-50ac-46dc-9ab4-1e8086ce64cb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:33.113087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-03955154-50ac-46dc-9ab4-1e8086ce64cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:33.172304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1044/2074] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:33.182578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:33.184039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:33.248906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b2c0c-6f53-4713-ae53-b075189b3e1f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:33.252571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db9b19-7881-4c61-ae96-0ab8d0ceb221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:33.253216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db9b19-7881-4c61-ae96-0ab8d0ceb221 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:33.253498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db9b19-7881-4c61-ae96-0ab8d0ceb221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:33.253771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db9b19-7881-4c61-ae96-0ab8d0ceb221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:33.264772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db9b19-7881-4c61-ae96-0ab8d0ceb221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:33.266200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db9b19-7881-4c61-ae96-0ab8d0ceb221 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:33.276053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1031/2075] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:33.830052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03955154-50ac-46dc-9ab4-1e8086ce64cb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:33.831500 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-03955154-50ac-46dc-9ab4-1e8086ce64cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:33.838565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:33.844663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03955154-50ac-46dc-9ab4-1e8086ce64cb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:33.849352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:33.850217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:33.850506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:33.850787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] PATH_INFO: `/v3/projects/c583ff5d62bd4bb0b090dbaae715274d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:33.851996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c583ff5d62bd4bb0b090dbaae715274d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:33.853902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:33.960494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c583ff5d62bd4bb0b090dbaae715274d', 'request_id': 'req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:33.966234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c583ff5d62bd4bb0b090dbaae715274d', 'request_id': 'req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.153929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c583ff5d62bd4bb0b090dbaae715274d', 'request_id': 'req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.168551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] The token cache is being invalidate because project c583ff5d62bd4bb0b090dbaae715274d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:34.168942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.169853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.171722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1045/2076] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:33 2022] DELETE /identity/v3/projects/c583ff5d62bd4bb0b090dbaae715274d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:34.176169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db9b19-7881-4c61-ae96-0ab8d0ceb221 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:34.186827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db9b19-7881-4c61-ae96-0ab8d0ceb221 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:34.206542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:34.247359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:34.248261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:34.251000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:34.251000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] PATH_INFO: `/v3/users/c97793c0f6d34022bbfdbe504b049c09` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:34.251000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c97793c0f6d34022bbfdbe504b049c09)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:34.252046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:34.454596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c97793c0f6d34022bbfdbe504b049c09'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.455274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c97793c0f6d34022bbfdbe504b049c09'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.456390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c97793c0f6d34022bbfdbe504b049c09'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.463240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c97793c0f6d34022bbfdbe504b049c09'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.475690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c97793c0f6d34022bbfdbe504b049c09'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:34.564020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1032/2077] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:34 2022] DELETE /identity/v3/users/c97793c0f6d34022bbfdbe504b049c09 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:35.023303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:35.025122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:35.037704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfbb3b-e7e3-4953-9b3a-be86cc086cec admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:35.105802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:35.106643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:35.106927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:35.107203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] PATH_INFO: `/v3/projects/ecd3bf6acc71496d929828817c7f6152` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:35.108416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd3bf6acc71496d929828817c7f6152)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:35.110318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:35.251805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd3bf6acc71496d929828817c7f6152', 'request_id': 'req-1ff57a72-5b9f-443d-9520-f8fb654b629f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.258483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd3bf6acc71496d929828817c7f6152', 'request_id': 'req-1ff57a72-5b9f-443d-9520-f8fb654b629f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.462247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd3bf6acc71496d929828817c7f6152', 'request_id': 'req-1ff57a72-5b9f-443d-9520-f8fb654b629f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.474675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] The token cache is being invalidate because project ecd3bf6acc71496d929828817c7f6152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:35.474968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.475626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.477203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1046/2078] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:35 2022] DELETE /identity/v3/projects/ecd3bf6acc71496d929828817c7f6152 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:35.481876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:35.483631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:35.498591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87252f77-9c06-4cb7-b95b-1b0b50d28b77 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:35.557498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:35.558701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:35.559099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:35.559244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] PATH_INFO: `/v3/users/ee94cd918269404e83657abf4467c8a3` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:35.560520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee94cd918269404e83657abf4467c8a3)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:35.562557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:35.805283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:35.807828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee94cd918269404e83657abf4467c8a3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.808498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee94cd918269404e83657abf4467c8a3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.809120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee94cd918269404e83657abf4467c8a3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.815773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee94cd918269404e83657abf4467c8a3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.828946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b01e4e96-47bf-412f-a1e8-04d748fe4ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee94cd918269404e83657abf4467c8a3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:35.918921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1033/2079] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:35 2022] DELETE /identity/v3/users/ee94cd918269404e83657abf4467c8a3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:36.385529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:36.387087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:36.401950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff57a72-5b9f-443d-9520-f8fb654b629f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:36.414555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:36.467690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:36.468564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:36.468873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:36.469177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] PATH_INFO: `/v3/projects/04c265572ac34a34a020d599d4b87f63` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:36.470410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c265572ac34a34a020d599d4b87f63)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:36.472397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:36.607183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c265572ac34a34a020d599d4b87f63', 'request_id': 'req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:36.612696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c265572ac34a34a020d599d4b87f63', 'request_id': 'req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:36.725445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c265572ac34a34a020d599d4b87f63', 'request_id': 'req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:36.737430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] The token cache is being invalidate because project 04c265572ac34a34a020d599d4b87f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:36.737727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:36.738394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dfed2f00-0c47-41f8-bfb4-cf2cca53353e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:36.739972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1047/2080] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:36 2022] DELETE /identity/v3/projects/04c265572ac34a34a020d599d4b87f63 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:36.745744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fc641-734f-43a5-8080-0a17bde67c06 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:36.746218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fc641-734f-43a5-8080-0a17bde67c06 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:36.746347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fc641-734f-43a5-8080-0a17bde67c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:36.766991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f08fc641-734f-43a5-8080-0a17bde67c06 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:36.768294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f08fc641-734f-43a5-8080-0a17bde67c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:36.837042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1034/2081] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:36.845930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce26fae-e47e-4c07-8cd1-34267ad2ad96 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:36.846246 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce26fae-e47e-4c07-8cd1-34267ad2ad96 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:36.846667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce26fae-e47e-4c07-8cd1-34267ad2ad96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:36.867081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dce26fae-e47e-4c07-8cd1-34267ad2ad96 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:36.868870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dce26fae-e47e-4c07-8cd1-34267ad2ad96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:36.879558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1048/2082] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:36.883632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fc641-734f-43a5-8080-0a17bde67c06 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:36.905566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fc641-734f-43a5-8080-0a17bde67c06 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:36.910776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:36.911485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:36.911774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:36.912054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:36.912799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:27:36.913108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:36.914785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:36.929860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1035/2083] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:27:36.932140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce26fae-e47e-4c07-8cd1-34267ad2ad96 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:36.942249 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce26fae-e47e-4c07-8cd1-34267ad2ad96 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:36.946979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cee079-12cb-41d3-84aa-0e5f37b2c84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:36.947667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cee079-12cb-41d3-84aa-0e5f37b2c84a admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:36.947954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cee079-12cb-41d3-84aa-0e5f37b2c84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:36.948231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cee079-12cb-41d3-84aa-0e5f37b2c84a admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:36.948931 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cee079-12cb-41d3-84aa-0e5f37b2c84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:36.955826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cee079-12cb-41d3-84aa-0e5f37b2c84a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:37.064909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1049/2084] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:36 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:27:37.070027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:37.081172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4b902-089f-4929-b326-465defcd6a9e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:37.136685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1189b-6bc1-4d61-a4b7-4d6df1197caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:37.137280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1189b-6bc1-4d61-a4b7-4d6df1197caf admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:37.137567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1189b-6bc1-4d61-a4b7-4d6df1197caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:37.137845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1189b-6bc1-4d61-a4b7-4d6df1197caf admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:37.138562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1189b-6bc1-4d61-a4b7-4d6df1197caf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:37.140508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1189b-6bc1-4d61-a4b7-4d6df1197caf admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:37.247662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1036/2085] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:37 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:27:37.251836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cee079-12cb-41d3-84aa-0e5f37b2c84a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:37.266612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cee079-12cb-41d3-84aa-0e5f37b2c84a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:37.272217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a8cc-7207-4319-8702-d8b50e6101cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:37.272929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a8cc-7207-4319-8702-d8b50e6101cc admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:37.273326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a8cc-7207-4319-8702-d8b50e6101cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:37.273608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a8cc-7207-4319-8702-d8b50e6101cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:37.274338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a8cc-7207-4319-8702-d8b50e6101cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:37.276000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a8cc-7207-4319-8702-d8b50e6101cc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:37.286871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1050/2086] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:37.292717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1189b-6bc1-4d61-a4b7-4d6df1197caf admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:37.302994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1189b-6bc1-4d61-a4b7-4d6df1197caf admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:37.307933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127c428-9ed7-4361-a1e8-a6b25daf528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:37.308581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127c428-9ed7-4361-a1e8-a6b25daf528c admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:37.308862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127c428-9ed7-4361-a1e8-a6b25daf528c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:37.309134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127c428-9ed7-4361-a1e8-a6b25daf528c admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:37.309850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127c428-9ed7-4361-a1e8-a6b25daf528c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:37.311485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127c428-9ed7-4361-a1e8-a6b25daf528c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:37.323524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1037/2087] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:37.327745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a8cc-7207-4319-8702-d8b50e6101cc admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:37.339924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a8cc-7207-4319-8702-d8b50e6101cc admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:37.344606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:37.345502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:37.345782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:37.346061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] PATH_INFO: `/v3/projects/890e7e79ff744b6b9589f166cd7d8979/users/9dc29ac59eed483b8c6443c397505adc/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:37.361241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=890e7e79ff744b6b9589f166cd7d8979, user_id=9dc29ac59eed483b8c6443c397505adc, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:37.363946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:37.440267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1051/2088] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:37 2022] PUT /identity/v3/projects/890e7e79ff744b6b9589f166cd7d8979/users/9dc29ac59eed483b8c6443c397505adc/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:37.446309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dee6d6-4d1e-4229-afe1-8122ccf4be48 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:37.446602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dee6d6-4d1e-4229-afe1-8122ccf4be48 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:37.446881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dee6d6-4d1e-4229-afe1-8122ccf4be48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:37.465539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-47dee6d6-4d1e-4229-afe1-8122ccf4be48 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:37.466780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-47dee6d6-4d1e-4229-afe1-8122ccf4be48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:37.488129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:37.525987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1038/2089] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:37.539609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:37.541527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:37.589615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:37.601667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0fa5e-fb97-4547-b65d-0ae581fab1bf admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:37.606182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:37.606889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:37.607283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:37.607771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:37.618596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:37.620218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:37.630502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1052/2090] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:39.024204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b732830-b517-4401-9593-54c53e5082b7 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:39.024764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b732830-b517-4401-9593-54c53e5082b7 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:39.024909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b732830-b517-4401-9593-54c53e5082b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:39.056409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7b732830-b517-4401-9593-54c53e5082b7 None None] MFA Rules not processed for user `9dc29ac59eed483b8c6443c397505adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:39.057632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7b732830-b517-4401-9593-54c53e5082b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:39.127713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1039/2091] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:27:39.138842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:39.140380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:39.206050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514baa3-52ea-4908-8b19-53d55aac12d3 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:39.209725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709b1ba-a871-4eb4-a9ef-b30173c23a4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:39.210378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709b1ba-a871-4eb4-a9ef-b30173c23a4e service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:39.215880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709b1ba-a871-4eb4-a9ef-b30173c23a4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:39.215880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709b1ba-a871-4eb4-a9ef-b30173c23a4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:39.222452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709b1ba-a871-4eb4-a9ef-b30173c23a4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:39.224600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709b1ba-a871-4eb4-a9ef-b30173c23a4e service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:39.238645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1053/2092] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:39.484932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b732830-b517-4401-9593-54c53e5082b7 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:39.486358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7b732830-b517-4401-9593-54c53e5082b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:39.500464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b732830-b517-4401-9593-54c53e5082b7 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:39.504945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:39.505900 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:39.506187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:39.506467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] PATH_INFO: `/v3/users/f1984ac56bc84ae29499a95f9b444bc0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:39.507707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1984ac56bc84ae29499a95f9b444bc0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:39.509609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:39.743985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1984ac56bc84ae29499a95f9b444bc0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:39.744627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1984ac56bc84ae29499a95f9b444bc0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:39.745221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1984ac56bc84ae29499a95f9b444bc0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:39.752063 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1984ac56bc84ae29499a95f9b444bc0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:39.764917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1984ac56bc84ae29499a95f9b444bc0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:39.880821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1040/2093] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:39 2022] DELETE /identity/v3/users/f1984ac56bc84ae29499a95f9b444bc0 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:27:39.885311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51dd97-056f-4b5f-a2a0-fec58f98de32 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:39.885607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51dd97-056f-4b5f-a2a0-fec58f98de32 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:39.885967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51dd97-056f-4b5f-a2a0-fec58f98de32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:39.904805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ee51dd97-056f-4b5f-a2a0-fec58f98de32 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:39.906073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ee51dd97-056f-4b5f-a2a0-fec58f98de32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:39.971431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1054/2094] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:39.983686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:39.985168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:40.026470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:40.052879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b135711-5bc6-44f7-aa9c-df415c9dc15f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:40.056625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa3bc1-ee80-42a9-8449-94549f56e91f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:40.057290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3bc1-ee80-42a9-8449-94549f56e91f service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:40.057555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3bc1-ee80-42a9-8449-94549f56e91f service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:40.057846 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa3bc1-ee80-42a9-8449-94549f56e91f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:40.067095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa3bc1-ee80-42a9-8449-94549f56e91f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:40.069109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa3bc1-ee80-42a9-8449-94549f56e91f service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:40.078621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1041/2095] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:40.624050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51dd97-056f-4b5f-a2a0-fec58f98de32 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:40.625584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ee51dd97-056f-4b5f-a2a0-fec58f98de32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:40.639082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51dd97-056f-4b5f-a2a0-fec58f98de32 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:40.644037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:40.645040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:40.645490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:40.645940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] PATH_INFO: `/v3/projects/1354b2d4544c487489ab820e9793e60a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:40.647350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1354b2d4544c487489ab820e9793e60a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:40.649913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:40.774612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1354b2d4544c487489ab820e9793e60a', 'request_id': 'req-87d9bdfd-c4fb-473c-b7c8-89704b168a24'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:40.779906 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1354b2d4544c487489ab820e9793e60a', 'request_id': 'req-87d9bdfd-c4fb-473c-b7c8-89704b168a24'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:40.879519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1354b2d4544c487489ab820e9793e60a', 'request_id': 'req-87d9bdfd-c4fb-473c-b7c8-89704b168a24'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:40.891008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] The token cache is being invalidate because project 1354b2d4544c487489ab820e9793e60a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:40.891324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:40.891945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-87d9bdfd-c4fb-473c-b7c8-89704b168a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:40.893507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1055/2096] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:40 2022] DELETE /identity/v3/projects/1354b2d4544c487489ab820e9793e60a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:40.898367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd17f24-18e9-4a4c-bb3f-050c12a7b2b0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:40.898682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd17f24-18e9-4a4c-bb3f-050c12a7b2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:40.898964 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd17f24-18e9-4a4c-bb3f-050c12a7b2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:40.916899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8bd17f24-18e9-4a4c-bb3f-050c12a7b2b0 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:40.918078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8bd17f24-18e9-4a4c-bb3f-050c12a7b2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:40.978467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1042/2097] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:40.986280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186346c-03c7-4010-b524-faae219688b2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:40.986576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186346c-03c7-4010-b524-faae219688b2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:40.986934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186346c-03c7-4010-b524-faae219688b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.007795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1186346c-03c7-4010-b524-faae219688b2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:41.009018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1186346c-03c7-4010-b524-faae219688b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:41.017072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1056/2098] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:41.021851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd17f24-18e9-4a4c-bb3f-050c12a7b2b0 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:41.032588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd17f24-18e9-4a4c-bb3f-050c12a7b2b0 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:41.037927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740f299-d019-4913-ae82-a46ac5a705b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:41.038582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f299-d019-4913-ae82-a46ac5a705b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.038865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f299-d019-4913-ae82-a46ac5a705b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.039149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740f299-d019-4913-ae82-a46ac5a705b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.039952 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740f299-d019-4913-ae82-a46ac5a705b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:27:41.040262 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740f299-d019-4913-ae82-a46ac5a705b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:41.041862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740f299-d019-4913-ae82-a46ac5a705b3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:41.060714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1043/2099] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:27:41.065030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186346c-03c7-4010-b524-faae219688b2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:41.073327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:41.077373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186346c-03c7-4010-b524-faae219688b2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:41.082168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cca5ab-11c6-4ad1-a8cb-d28a781eae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:41.082972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cca5ab-11c6-4ad1-a8cb-d28a781eae93 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.083801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cca5ab-11c6-4ad1-a8cb-d28a781eae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.084022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cabce-b4cb-442e-82a2-c16e664f831b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.084165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cca5ab-11c6-4ad1-a8cb-d28a781eae93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.084310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cabce-b4cb-442e-82a2-c16e664f831b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.084591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cabce-b4cb-442e-82a2-c16e664f831b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.084868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cca5ab-11c6-4ad1-a8cb-d28a781eae93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:41.086758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cca5ab-11c6-4ad1-a8cb-d28a781eae93 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:41.103149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6c0cabce-b4cb-442e-82a2-c16e664f831b None None] MFA Rules not processed for user `9dc29ac59eed483b8c6443c397505adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:41.104412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6c0cabce-b4cb-442e-82a2-c16e664f831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:41.157624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1044/2100] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:27:41.225818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1057/2101] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:41 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:27:41.231004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cabce-b4cb-442e-82a2-c16e664f831b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:41.243381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cabce-b4cb-442e-82a2-c16e664f831b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:41.308159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6f53e-3f60-4114-9f32-c750c9921161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:41.308756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6f53e-3f60-4114-9f32-c750c9921161 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.309089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6f53e-3f60-4114-9f32-c750c9921161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.309369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6f53e-3f60-4114-9f32-c750c9921161 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.310076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6f53e-3f60-4114-9f32-c750c9921161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:41.312134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6f53e-3f60-4114-9f32-c750c9921161 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:41.322192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:41.483092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1045/2102] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:41 2022] POST /identity/v3/users => generated 287 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:41.484690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cca5ab-11c6-4ad1-a8cb-d28a781eae93 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:41.495816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cca5ab-11c6-4ad1-a8cb-d28a781eae93 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:41.500745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416c29e-6e97-438b-9c07-13205206fa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:41.501406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416c29e-6e97-438b-9c07-13205206fa9c admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.501698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416c29e-6e97-438b-9c07-13205206fa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.501975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416c29e-6e97-438b-9c07-13205206fa9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.502709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416c29e-6e97-438b-9c07-13205206fa9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:41.504362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416c29e-6e97-438b-9c07-13205206fa9c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:41.519158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1058/2103] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:41.520164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6f53e-3f60-4114-9f32-c750c9921161 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:41.531566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6f53e-3f60-4114-9f32-c750c9921161 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:41.537027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159178e8-8bb2-4858-b1d6-36cd52ee2b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:41.537799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159178e8-8bb2-4858-b1d6-36cd52ee2b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.538156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159178e8-8bb2-4858-b1d6-36cd52ee2b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.538515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159178e8-8bb2-4858-b1d6-36cd52ee2b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.539364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159178e8-8bb2-4858-b1d6-36cd52ee2b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:41.541239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159178e8-8bb2-4858-b1d6-36cd52ee2b8f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:41.556963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1046/2104] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:41.558103 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416c29e-6e97-438b-9c07-13205206fa9c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:41.569404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416c29e-6e97-438b-9c07-13205206fa9c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:41.574054 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46184d60-ac77-4e4a-b05b-b35d161696ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:41.574977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46184d60-ac77-4e4a-b05b-b35d161696ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.575299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46184d60-ac77-4e4a-b05b-b35d161696ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.575736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46184d60-ac77-4e4a-b05b-b35d161696ff admin admin] PATH_INFO: `/v3/projects/7649f40ca00d4d6b8faadb7581da9ec4/users/0e593b0cc3644f1784160e2b56205d33/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.598107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46184d60-ac77-4e4a-b05b-b35d161696ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7649f40ca00d4d6b8faadb7581da9ec4, user_id=0e593b0cc3644f1784160e2b56205d33, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:41.601859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46184d60-ac77-4e4a-b05b-b35d161696ff admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:41.726458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1059/2105] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:41 2022] PUT /identity/v3/projects/7649f40ca00d4d6b8faadb7581da9ec4/users/0e593b0cc3644f1784160e2b56205d33/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:27:41.737604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429e421-cb9a-4218-adb2-f10428b26e05 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.738054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429e421-cb9a-4218-adb2-f10428b26e05 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.740824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429e421-cb9a-4218-adb2-f10428b26e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.769190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0429e421-cb9a-4218-adb2-f10428b26e05 None None] MFA Rules not processed for user `0e593b0cc3644f1784160e2b56205d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:41.770559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0429e421-cb9a-4218-adb2-f10428b26e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:41.837183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1047/2106] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:41.841993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46184d60-ac77-4e4a-b05b-b35d161696ff admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:41.918397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46184d60-ac77-4e4a-b05b-b35d161696ff admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:41.923486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f43f4-cc75-4f34-9b4b-797d9a928904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:41.924278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f43f4-cc75-4f34-9b4b-797d9a928904 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:41.924565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f43f4-cc75-4f34-9b4b-797d9a928904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:41.924840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f43f4-cc75-4f34-9b4b-797d9a928904 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:41.925934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f43f4-cc75-4f34-9b4b-797d9a928904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:41.928019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03f43f4-cc75-4f34-9b4b-797d9a928904 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:42.161591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1060/2107] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:41 2022] POST /identity/v3/projects => generated 377 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:27:42.165611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429e421-cb9a-4218-adb2-f10428b26e05 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:42.176235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429e421-cb9a-4218-adb2-f10428b26e05 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:42.232264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b965765-cb29-4fcc-951a-b808e3e3e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:42.232930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b965765-cb29-4fcc-951a-b808e3e3e341 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:42.233218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b965765-cb29-4fcc-951a-b808e3e3e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:42.233495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b965765-cb29-4fcc-951a-b808e3e3e341 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:42.234203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b965765-cb29-4fcc-951a-b808e3e3e341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:42.236498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b965765-cb29-4fcc-951a-b808e3e3e341 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:42.337082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1048/2108] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:42 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:42.340937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f43f4-cc75-4f34-9b4b-797d9a928904 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:42.351999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f43f4-cc75-4f34-9b4b-797d9a928904 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:42.356672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b3fc9-7bec-4085-a6f2-f6af386cb812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:42.357341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b3fc9-7bec-4085-a6f2-f6af386cb812 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:42.357628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b3fc9-7bec-4085-a6f2-f6af386cb812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:42.357907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b3fc9-7bec-4085-a6f2-f6af386cb812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:42.358641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b3fc9-7bec-4085-a6f2-f6af386cb812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:42.360290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b3fc9-7bec-4085-a6f2-f6af386cb812 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:42.371647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1061/2109] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:42.375811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b965765-cb29-4fcc-951a-b808e3e3e341 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:42.386500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b965765-cb29-4fcc-951a-b808e3e3e341 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:42.391205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d15491-16f8-468e-bb67-f9480fbf4f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:42.392078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d15491-16f8-468e-bb67-f9480fbf4f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:42.392359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d15491-16f8-468e-bb67-f9480fbf4f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:42.392636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d15491-16f8-468e-bb67-f9480fbf4f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:42.393385 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d15491-16f8-468e-bb67-f9480fbf4f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:42.395226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d15491-16f8-468e-bb67-f9480fbf4f43 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:42.408007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1049/2110] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:42.412148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b3fc9-7bec-4085-a6f2-f6af386cb812 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:42.422142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b3fc9-7bec-4085-a6f2-f6af386cb812 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:42.426738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a027747-ddc3-424a-a9ed-208896bc4ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:42.427936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027747-ddc3-424a-a9ed-208896bc4ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:42.428222 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027747-ddc3-424a-a9ed-208896bc4ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:42.428502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027747-ddc3-424a-a9ed-208896bc4ec9 admin admin] PATH_INFO: `/v3/projects/cee07e14354e4637954d58efe233e816/users/60733437cef44dbabfcb50d53a92118c/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:42.443493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a027747-ddc3-424a-a9ed-208896bc4ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee07e14354e4637954d58efe233e816, user_id=60733437cef44dbabfcb50d53a92118c, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:42.446136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a027747-ddc3-424a-a9ed-208896bc4ec9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:42.555179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1062/2111] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:42 2022] PUT /identity/v3/projects/cee07e14354e4637954d58efe233e816/users/60733437cef44dbabfcb50d53a92118c/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:42.565455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c559f-cc1c-4443-b427-949b0ac63983 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:42.565752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c559f-cc1c-4443-b427-949b0ac63983 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:42.566033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c559f-cc1c-4443-b427-949b0ac63983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:42.594883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a61c559f-cc1c-4443-b427-949b0ac63983 None None] MFA Rules not processed for user `60733437cef44dbabfcb50d53a92118c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:42.601465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a61c559f-cc1c-4443-b427-949b0ac63983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:42.645716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:42.669088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1050/2112] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:42.673536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a027747-ddc3-424a-a9ed-208896bc4ec9 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:42.734147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:42.738053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a027747-ddc3-424a-a9ed-208896bc4ec9 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:42.743283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335af70c-3894-4c86-bb54-e542da575e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:42.743952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335af70c-3894-4c86-bb54-e542da575e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:42.744281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335af70c-3894-4c86-bb54-e542da575e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:42.744561 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335af70c-3894-4c86-bb54-e542da575e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:42.745270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335af70c-3894-4c86-bb54-e542da575e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:42.747157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335af70c-3894-4c86-bb54-e542da575e16 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:42.848854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1063/2113] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:27:42 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:27:42.866781 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c559f-cc1c-4443-b427-949b0ac63983 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:42.878048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c559f-cc1c-4443-b427-949b0ac63983 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:42.958086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba451cf2-7079-460d-8c56-0a1b54d672de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:42.958086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451cf2-7079-460d-8c56-0a1b54d672de admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:42.958086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451cf2-7079-460d-8c56-0a1b54d672de admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:42.958086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451cf2-7079-460d-8c56-0a1b54d672de admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:42.958426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba451cf2-7079-460d-8c56-0a1b54d672de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:42.958426 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba451cf2-7079-460d-8c56-0a1b54d672de admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:43.086179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1051/2114] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:27:42 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:27:43.090372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335af70c-3894-4c86-bb54-e542da575e16 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:43.101645 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335af70c-3894-4c86-bb54-e542da575e16 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:43.106400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e4c3-7201-4be2-91e3-dffbde01bcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:43.107068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e4c3-7201-4be2-91e3-dffbde01bcdf admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:43.107396 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e4c3-7201-4be2-91e3-dffbde01bcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:43.107676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6e4c3-7201-4be2-91e3-dffbde01bcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:43.108443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6e4c3-7201-4be2-91e3-dffbde01bcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:43.110076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6e4c3-7201-4be2-91e3-dffbde01bcdf admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:43.121988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1064/2115] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:43.126186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba451cf2-7079-460d-8c56-0a1b54d672de admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:43.137244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba451cf2-7079-460d-8c56-0a1b54d672de admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:43.142782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f672e7-f257-442d-9f09-b2ab67137338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:43.143764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f672e7-f257-442d-9f09-b2ab67137338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:43.143973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f672e7-f257-442d-9f09-b2ab67137338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:43.144252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f672e7-f257-442d-9f09-b2ab67137338 admin admin] PATH_INFO: `/v3/projects/1fc8f50e6be4422982d9e34416f3a1b1/users/03ff545c5c17447f8f14dbae2a82a855/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:43.163522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f672e7-f257-442d-9f09-b2ab67137338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc8f50e6be4422982d9e34416f3a1b1, user_id=03ff545c5c17447f8f14dbae2a82a855, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:43.166219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f672e7-f257-442d-9f09-b2ab67137338 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:43.255057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1052/2116] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:43 2022] PUT /identity/v3/projects/1fc8f50e6be4422982d9e34416f3a1b1/users/03ff545c5c17447f8f14dbae2a82a855/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:27:43.258985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e4c3-7201-4be2-91e3-dffbde01bcdf admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:43.269079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6e4c3-7201-4be2-91e3-dffbde01bcdf admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:43.327775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836d818-6aa8-4139-baf3-dbc0db0c0457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:43.328442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836d818-6aa8-4139-baf3-dbc0db0c0457 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:43.328724 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836d818-6aa8-4139-baf3-dbc0db0c0457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:43.329000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836d818-6aa8-4139-baf3-dbc0db0c0457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:43.329723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836d818-6aa8-4139-baf3-dbc0db0c0457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:43.331391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836d818-6aa8-4139-baf3-dbc0db0c0457 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:43.342089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1065/2117] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:27:43.346427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f672e7-f257-442d-9f09-b2ab67137338 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:43.360152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f672e7-f257-442d-9f09-b2ab67137338 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:43.365157 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:43.366107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:43.366529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:43.366821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] PATH_INFO: `/v3/projects/1fc8f50e6be4422982d9e34416f3a1b1/users/03ff545c5c17447f8f14dbae2a82a855/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:43.368936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc8f50e6be4422982d9e34416f3a1b1, user_id=03ff545c5c17447f8f14dbae2a82a855, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:43.371720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:43.390409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1053/2118] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:27:43 2022] PUT /identity/v3/projects/1fc8f50e6be4422982d9e34416f3a1b1/users/03ff545c5c17447f8f14dbae2a82a855/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:43.393518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe44a-04dc-4850-96fd-4ee201696bd1 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:43.393812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe44a-04dc-4850-96fd-4ee201696bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:43.394116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe44a-04dc-4850-96fd-4ee201696bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:43.421516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-202fe44a-04dc-4850-96fd-4ee201696bd1 None None] MFA Rules not processed for user `03ff545c5c17447f8f14dbae2a82a855`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:43.422737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-202fe44a-04dc-4850-96fd-4ee201696bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:43.488359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1066/2119] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:43.499334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:43.501005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:43.558378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e2f9d-25b8-4896-b031-af947ce8a2b9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:43.566733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc7aa2-84cc-40d2-9638-6b459a73d0c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:43.567652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7aa2-84cc-40d2-9638-6b459a73d0c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:43.568020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7aa2-84cc-40d2-9638-6b459a73d0c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:43.568300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc7aa2-84cc-40d2-9638-6b459a73d0c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:43.624191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:43.638030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc7aa2-84cc-40d2-9638-6b459a73d0c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:43.639752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc7aa2-84cc-40d2-9638-6b459a73d0c9 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:43.649246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1054/2120] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:45.757656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fe44a-04dc-4850-96fd-4ee201696bd1 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:45.769998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fe44a-04dc-4850-96fd-4ee201696bd1 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:45.778064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af5094-7a6c-4ff8-93c7-e5532f84e004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:45.778064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af5094-7a6c-4ff8-93c7-e5532f84e004 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:45.778064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af5094-7a6c-4ff8-93c7-e5532f84e004 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:45.778064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af5094-7a6c-4ff8-93c7-e5532f84e004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:45.832946 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:45.843041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af5094-7a6c-4ff8-93c7-e5532f84e004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:45.844633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af5094-7a6c-4ff8-93c7-e5532f84e004 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:45.854139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1067/2121] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:47.749685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc7aa2-84cc-40d2-9638-6b459a73d0c9 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:47.759125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc7aa2-84cc-40d2-9638-6b459a73d0c9 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:47.762897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31103dd9-675b-4ee8-8932-436f1c1cd60b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:47.763660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31103dd9-675b-4ee8-8932-436f1c1cd60b service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:47.763949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31103dd9-675b-4ee8-8932-436f1c1cd60b service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:47.764299 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31103dd9-675b-4ee8-8932-436f1c1cd60b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:47.776093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31103dd9-675b-4ee8-8932-436f1c1cd60b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:47.778044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31103dd9-675b-4ee8-8932-436f1c1cd60b service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:47.789103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1055/2122] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:53.079594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af5094-7a6c-4ff8-93c7-e5532f84e004 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:53.158431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af5094-7a6c-4ff8-93c7-e5532f84e004 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:53.164785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:53.169849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:53.169849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:53.169849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] PATH_INFO: `/v3/users/0e593b0cc3644f1784160e2b56205d33` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:53.169849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e593b0cc3644f1784160e2b56205d33)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:53.170857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:53.448610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e593b0cc3644f1784160e2b56205d33'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:53.449530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e593b0cc3644f1784160e2b56205d33'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:53.450187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e593b0cc3644f1784160e2b56205d33'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:53.458315 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e593b0cc3644f1784160e2b56205d33'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:53.473041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e593b0cc3644f1784160e2b56205d33'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:53.601682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1068/2123] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:53 2022] DELETE /identity/v3/users/0e593b0cc3644f1784160e2b56205d33 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:53.606622 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e4df-4f7e-47fc-a266-4ca708ca2c95 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:53.610999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e4df-4f7e-47fc-a266-4ca708ca2c95 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:53.610999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e4df-4f7e-47fc-a266-4ca708ca2c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:53.626688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7f86e4df-4f7e-47fc-a266-4ca708ca2c95 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:53.627856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7f86e4df-4f7e-47fc-a266-4ca708ca2c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:53.692759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1056/2124] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:53.703034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:53.704577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:53.764486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecaab8-f066-475e-80bc-41ec8890f3c5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:53.768244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:53.768898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:53.771521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:53.771855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:53.781988 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:53.783460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:53.793805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1069/2125] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:27:54.216765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86e4df-4f7e-47fc-a266-4ca708ca2c95 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:54.218265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7f86e4df-4f7e-47fc-a266-4ca708ca2c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:54.230590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86e4df-4f7e-47fc-a266-4ca708ca2c95 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:54.235235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:54.239002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:54.239002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:54.239002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] PATH_INFO: `/v3/users/fd8707405a044f0a991a3129739f1207` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:54.239002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd8707405a044f0a991a3129739f1207)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:54.239955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:54.390411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:54.391930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:54.405849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8f079-5ef7-4c47-8f56-8732cea901ba service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:54.410040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:54.410903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:54.411216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:54.411525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] PATH_INFO: `/v3/projects/7649f40ca00d4d6b8faadb7581da9ec4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:54.412712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7649f40ca00d4d6b8faadb7581da9ec4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:54.414609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:54.422970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:54.460320 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd8707405a044f0a991a3129739f1207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.461013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd8707405a044f0a991a3129739f1207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.461664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd8707405a044f0a991a3129739f1207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.468790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd8707405a044f0a991a3129739f1207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.480971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5ee0403f-1098-4b50-96b1-e7b274522d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd8707405a044f0a991a3129739f1207'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.530918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7649f40ca00d4d6b8faadb7581da9ec4', 'request_id': 'req-a340df6a-5b68-4917-8cae-54c09415ead7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.536403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7649f40ca00d4d6b8faadb7581da9ec4', 'request_id': 'req-a340df6a-5b68-4917-8cae-54c09415ead7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.549018 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1057/2126] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:54 2022] DELETE /identity/v3/users/fd8707405a044f0a991a3129739f1207 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:54.558276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01245f-c1ae-4ab5-bd9c-a8342e65732b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:54.558568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01245f-c1ae-4ab5-bd9c-a8342e65732b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:54.558851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d01245f-c1ae-4ab5-bd9c-a8342e65732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:54.584223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8d01245f-c1ae-4ab5-bd9c-a8342e65732b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:27:54.585367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8d01245f-c1ae-4ab5-bd9c-a8342e65732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:54.634970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:54.636106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7649f40ca00d4d6b8faadb7581da9ec4', 'request_id': 'req-a340df6a-5b68-4917-8cae-54c09415ead7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.654654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1058/2127] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:27:54.665585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01245f-c1ae-4ab5-bd9c-a8342e65732b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:54.666983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8d01245f-c1ae-4ab5-bd9c-a8342e65732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:54.669557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] The token cache is being invalidate because project 7649f40ca00d4d6b8faadb7581da9ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:54.669959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.670677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:54.673105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1070/2128] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:54 2022] DELETE /identity/v3/projects/7649f40ca00d4d6b8faadb7581da9ec4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:27:54.678629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:54.680043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:54.737726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d01245f-c1ae-4ab5-bd9c-a8342e65732b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:54.742347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:54.743073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:54.743458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:54.743800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:54.745714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:54.764104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:54.765685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:54.766344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340df6a-5b68-4917-8cae-54c09415ead7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:54.774121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:54.775913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:54.776270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:54.776620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] PATH_INFO: `/v3/users/60733437cef44dbabfcb50d53a92118c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:54.777373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1059/2129] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:27:54.778919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60733437cef44dbabfcb50d53a92118c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:54.782206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:55.051883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60733437cef44dbabfcb50d53a92118c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.052649 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60733437cef44dbabfcb50d53a92118c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.053693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60733437cef44dbabfcb50d53a92118c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.061950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60733437cef44dbabfcb50d53a92118c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.074779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60733437cef44dbabfcb50d53a92118c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.166172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1071/2130] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:54 2022] DELETE /identity/v3/users/60733437cef44dbabfcb50d53a92118c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:55.293497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:55.297322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:55.311620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220593e-d6dd-408f-b882-594cb1ae0828 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:55.370057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:55.370932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:55.371210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:55.371512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] PATH_INFO: `/v3/projects/960d84c221fb4a1eabc3a6d1ae34a97e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:55.381581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=960d84c221fb4a1eabc3a6d1ae34a97e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:55.389000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:55.549183 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960d84c221fb4a1eabc3a6d1ae34a97e', 'request_id': 'req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.556341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960d84c221fb4a1eabc3a6d1ae34a97e', 'request_id': 'req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.811246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960d84c221fb4a1eabc3a6d1ae34a97e', 'request_id': 'req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.822635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] The token cache is being invalidate because project 960d84c221fb4a1eabc3a6d1ae34a97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:55.823508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.823620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:55.825084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1060/2131] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:55 2022] DELETE /identity/v3/projects/960d84c221fb4a1eabc3a6d1ae34a97e => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:55.861461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:55.863178 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:55.882719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbea00-89f6-4a66-90e9-a64033e19e60 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:55.907735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:55.957731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:55.958893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:55.959190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:55.959521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] PATH_INFO: `/v3/projects/cee07e14354e4637954d58efe233e816` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:55.960900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee07e14354e4637954d58efe233e816)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:55.962948 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:56.107289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee07e14354e4637954d58efe233e816', 'request_id': 'req-d50e1143-3beb-4a8f-a585-b250af56b20b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.113950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee07e14354e4637954d58efe233e816', 'request_id': 'req-d50e1143-3beb-4a8f-a585-b250af56b20b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.228561 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee07e14354e4637954d58efe233e816', 'request_id': 'req-d50e1143-3beb-4a8f-a585-b250af56b20b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.240476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] The token cache is being invalidate because project cee07e14354e4637954d58efe233e816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:56.240786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.241421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.242982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1072/2132] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:55 2022] DELETE /identity/v3/projects/cee07e14354e4637954d58efe233e816 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:56.246979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:56.248479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:56.261242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e0de6-ebb9-4e0d-9035-cf17d02bdc29 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:56.281455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:56.327818 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:56.328624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:56.329098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:56.329380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] PATH_INFO: `/v3/users/03ff545c5c17447f8f14dbae2a82a855` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:56.330601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ff545c5c17447f8f14dbae2a82a855)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:56.332577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:56.568604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ff545c5c17447f8f14dbae2a82a855'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.569400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ff545c5c17447f8f14dbae2a82a855'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.570057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ff545c5c17447f8f14dbae2a82a855'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.577303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ff545c5c17447f8f14dbae2a82a855'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.592519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ff9982dd-2187-498a-8087-a64d06495bbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ff545c5c17447f8f14dbae2a82a855'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:56.674664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1061/2133] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:27:56 2022] DELETE /identity/v3/users/03ff545c5c17447f8f14dbae2a82a855 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:27:57.159571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:27:57.159571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:27:57.170418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e1143-3beb-4a8f-a585-b250af56b20b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:27:57.246491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:27:57.247406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:27:57.247693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:27:57.247972 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] PATH_INFO: `/v3/projects/1fc8f50e6be4422982d9e34416f3a1b1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:27:57.249226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc8f50e6be4422982d9e34416f3a1b1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:27:57.251183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:27:57.416731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:27:57.420458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc8f50e6be4422982d9e34416f3a1b1', 'request_id': 'req-121683d0-0d59-4b64-b183-24bdbc6891a9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:57.427347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc8f50e6be4422982d9e34416f3a1b1', 'request_id': 'req-121683d0-0d59-4b64-b183-24bdbc6891a9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:57.520556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc8f50e6be4422982d9e34416f3a1b1', 'request_id': 'req-121683d0-0d59-4b64-b183-24bdbc6891a9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:57.534718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] The token cache is being invalidate because project 1fc8f50e6be4422982d9e34416f3a1b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:27:57.535016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:57.537327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-121683d0-0d59-4b64-b183-24bdbc6891a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:27:57.540329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1073/2134] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:27:57 2022] DELETE /identity/v3/projects/1fc8f50e6be4422982d9e34416f3a1b1 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:28:06.038726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adce248-b219-4539-9821-93bd27539846 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:28:06.039295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adce248-b219-4539-9821-93bd27539846 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:28:06.039295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adce248-b219-4539-9821-93bd27539846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:28:06.067830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2adce248-b219-4539-9821-93bd27539846 None None] MFA Rules not processed for user `9dc29ac59eed483b8c6443c397505adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:28:06.070071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2adce248-b219-4539-9821-93bd27539846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:28:06.155141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1062/2135] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:28:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:28:06.508463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baf846-aa3d-4e29-9c17-4d474357b8b7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:28:06.509696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baf846-aa3d-4e29-9c17-4d474357b8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:28:06.510391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5baf846-aa3d-4e29-9c17-4d474357b8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:28:06.534171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b5baf846-aa3d-4e29-9c17-4d474357b8b7 None None] MFA Rules not processed for user `cbcc15c786fb43669ac34a4897735c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:28:06.535710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b5baf846-aa3d-4e29-9c17-4d474357b8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:28:06.602722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1074/2136] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:28:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:28:09.689533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e448d3-4dcc-4fd8-87a7-1999793bbe84 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:28:09.690064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e448d3-4dcc-4fd8-87a7-1999793bbe84 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:28:09.690216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e448d3-4dcc-4fd8-87a7-1999793bbe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:28:09.729347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-77e448d3-4dcc-4fd8-87a7-1999793bbe84 None None] MFA Rules not processed for user `e47c365946cf4264a2d2205ace04379d`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:28:09.729347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-77e448d3-4dcc-4fd8-87a7-1999793bbe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:28:09.769773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:28:09.789505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1063/2137] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:28:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:28:16.595339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=32,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:28:20.000421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:28:57.221144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fabbc7-f1a7-4848-a8eb-aa5969e08717 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:28:57.221684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fabbc7-f1a7-4848-a8eb-aa5969e08717 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:28:57.221781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fabbc7-f1a7-4848-a8eb-aa5969e08717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:28:57.239836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b6fabbc7-f1a7-4848-a8eb-aa5969e08717 None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:28:57.241041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b6fabbc7-f1a7-4848-a8eb-aa5969e08717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:28:57.302822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1075/2138] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:28:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:28:57.344602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e448d3-4dcc-4fd8-87a7-1999793bbe84 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:28:57.346664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-77e448d3-4dcc-4fd8-87a7-1999793bbe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:28:57.416609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e448d3-4dcc-4fd8-87a7-1999793bbe84 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:28:57.421491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f16e4-1654-4b1a-844e-e9af67a8ccbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:28:57.422163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f16e4-1654-4b1a-844e-e9af67a8ccbe service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:28:57.422446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f16e4-1654-4b1a-844e-e9af67a8ccbe service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:28:57.422730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f16e4-1654-4b1a-844e-e9af67a8ccbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:28:57.430897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f16e4-1654-4b1a-844e-e9af67a8ccbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:28:57.432812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f16e4-1654-4b1a-844e-e9af67a8ccbe service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:28:57.441127 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1064/2139] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 11:28:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:29:00.173229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fabbc7-f1a7-4848-a8eb-aa5969e08717 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:29:00.173229 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b6fabbc7-f1a7-4848-a8eb-aa5969e08717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:29:00.277580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fabbc7-f1a7-4848-a8eb-aa5969e08717 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:29:00.288051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc602bb-f20e-4861-abc4-f4d5c01d26e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:29:00.289076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc602bb-f20e-4861-abc4-f4d5c01d26e0 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:00.289582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc602bb-f20e-4861-abc4-f4d5c01d26e0 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:00.290076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc602bb-f20e-4861-abc4-f4d5c01d26e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:00.292905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2fc602bb-f20e-4861-abc4-f4d5c01d26e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:29:00.311905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc602bb-f20e-4861-abc4-f4d5c01d26e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:29:00.314143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc602bb-f20e-4861-abc4-f4d5c01d26e0 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:29:00.326945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1076/2140] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:29:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:29:07.437698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=32 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:29:10.321743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=57 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:29:18.565833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768cf33-f4a6-4061-b5d6-d3261dcee0a4 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:18.566432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768cf33-f4a6-4061-b5d6-d3261dcee0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:18.566432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768cf33-f4a6-4061-b5d6-d3261dcee0a4 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:18.567921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1065/2141] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:29:18.574001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1e29-a7c6-4706-9a24-e4001f4d893b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:18.574609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1e29-a7c6-4706-9a24-e4001f4d893b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:18.574609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f1e29-a7c6-4706-9a24-e4001f4d893b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:18.594048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dd2f1e29-a7c6-4706-9a24-e4001f4d893b None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:29:18.594899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dd2f1e29-a7c6-4706-9a24-e4001f4d893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:29:18.664520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1077/2142] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:29:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:29:18.677179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768cf33-f4a6-4061-b5d6-d3261dcee0a4 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:29:18.678960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b768cf33-f4a6-4061-b5d6-d3261dcee0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:29:18.740845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768cf33-f4a6-4061-b5d6-d3261dcee0a4 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:29:18.745906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7befeb-5741-4071-bbec-0c7d6ba5f718 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e939bc73c40140cb89517e918722d154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:29:18.746773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7befeb-5741-4071-bbec-0c7d6ba5f718 service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:18.748909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7befeb-5741-4071-bbec-0c7d6ba5f718 service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:18.748909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7befeb-5741-4071-bbec-0c7d6ba5f718 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:18.759957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7befeb-5741-4071-bbec-0c7d6ba5f718 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:29:18.761688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7befeb-5741-4071-bbec-0c7d6ba5f718 service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:29:18.778454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1066/2143] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:29:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:29:19.004312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89808aa3-a7d4-45d1-810b-743a5b81cda5 None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:19.004615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89808aa3-a7d4-45d1-810b-743a5b81cda5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:19.004899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89808aa3-a7d4-45d1-810b-743a5b81cda5 None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:19.006331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1078/2144] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:29:19.011811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab52520-147d-4f1e-8644-0bd01372dc57 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:19.012189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab52520-147d-4f1e-8644-0bd01372dc57 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:19.012797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab52520-147d-4f1e-8644-0bd01372dc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:19.031963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2ab52520-147d-4f1e-8644-0bd01372dc57 None None] MFA Rules not processed for user `cb1fa1e11b1a44aaa31cc19f7c199285`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:29:19.033284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2ab52520-147d-4f1e-8644-0bd01372dc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:29:19.095807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1067/2145] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:29:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:29:19.109062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89808aa3-a7d4-45d1-810b-743a5b81cda5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:29:19.111091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-89808aa3-a7d4-45d1-810b-743a5b81cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:29:19.125198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89808aa3-a7d4-45d1-810b-743a5b81cda5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:29:19.129031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a161193-aaef-4990-8566-d2118c030afb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:29:19.129656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a161193-aaef-4990-8566-d2118c030afb service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:19.129942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a161193-aaef-4990-8566-d2118c030afb service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:19.130209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a161193-aaef-4990-8566-d2118c030afb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:19.138458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a161193-aaef-4990-8566-d2118c030afb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:29:19.140812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a161193-aaef-4990-8566-d2118c030afb service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:29:19.150369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1079/2146] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:29:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:29:19.264394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a547c9-369b-4641-bb9a-4739cbed82eb None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:19.264702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a547c9-369b-4641-bb9a-4739cbed82eb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:19.264963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a547c9-369b-4641-bb9a-4739cbed82eb None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:19.266348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1068/2147] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:29:19.272148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff47de-98d2-47fc-b9f6-b7b0d12e8eec None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:19.272952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff47de-98d2-47fc-b9f6-b7b0d12e8eec None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:19.274259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff47de-98d2-47fc-b9f6-b7b0d12e8eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:19.297227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9aff47de-98d2-47fc-b9f6-b7b0d12e8eec None None] MFA Rules not processed for user `258ce5f4d6ff400199a80157cc34f5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:29:19.298539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9aff47de-98d2-47fc-b9f6-b7b0d12e8eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:29:19.316384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1080/2148] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:29:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:29:19.322837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a547c9-369b-4641-bb9a-4739cbed82eb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:29:19.325044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e1a547c9-369b-4641-bb9a-4739cbed82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:29:19.341686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a547c9-369b-4641-bb9a-4739cbed82eb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:29:19.345626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7dee1-5c29-4803-82d2-c3bcb98098e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:29:19.346528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7dee1-5c29-4803-82d2-c3bcb98098e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:29:19.346969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7dee1-5c29-4803-82d2-c3bcb98098e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:29:19.347437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7dee1-5c29-4803-82d2-c3bcb98098e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:29:19.365607 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7dee1-5c29-4803-82d2-c3bcb98098e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:29:19.367151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7dee1-5c29-4803-82d2-c3bcb98098e3 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:29:19.379501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1069/2149] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:29:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:29:29.290238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=40 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:29:29.374107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=55 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:31:13.317175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff47de-98d2-47fc-b9f6-b7b0d12e8eec None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:31:13.318582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9aff47de-98d2-47fc-b9f6-b7b0d12e8eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:13.373890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff47de-98d2-47fc-b9f6-b7b0d12e8eec None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:31:13.377416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:31:13.378115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:31:13.378426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:31:13.378702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:31:13.380498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:13.393550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:31:13.394975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:31:13.406139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1081/2150] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:31:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:31:13.413218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7dee1-5c29-4803-82d2-c3bcb98098e3 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:31:13.423175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7dee1-5c29-4803-82d2-c3bcb98098e3 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:31:13.426768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:31:13.427625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:31:13.427982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:31:13.428261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:31:13.430115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:13.441264 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:31:13.442662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:31:13.451711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1070/2151] 10.222.0.26 () {60 vars in 1396 bytes} [Mon Jul 11 11:31:13 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:31:13.499042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:31:13.512215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115131f4-3790-43f2-920e-98f228691346 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:31:13.512433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:31:13.513077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:31:13.513359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:31:13.513470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:31:13.513692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:31:13.514783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:13.515476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:13.529746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:31:13.530950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1e1c3-4098-4cd0-8732-bb90164c2c89 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:31:13.531168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:31:13.536575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:31:13.537212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:31:13.537495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:31:13.537773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:31:13.539574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:13.542661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1082/2152] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:31:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:31:13.549596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:31:13.557468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715569b-e0b8-4693-9170-a6d6f3d62e6b service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:31:13.561365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:31:13.561989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:31:13.562277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:31:13.562552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:31:13.564493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:13.598384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:31:13.600052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:31:13.609753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1071/2153] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:31:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:31:13.627661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:31:13.629095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:31:13.638817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1083/2154] 10.222.0.26 () {60 vars in 1396 bytes} [Mon Jul 11 11:31:13 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:31:13.790025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:31:13.800909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762682b-3694-434d-bd8f-b636719857b8 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:31:13.806579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:31:13.807227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:31:13.807546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:31:13.807826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:31:13.809637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:13.820416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:31:13.822503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:31:13.831733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1072/2155] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:31:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:31:23.634032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=60 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:31:23.827858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=40 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:31:47.984349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:31:47.994483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d605-794a-4710-8d2b-7c7fee6aaa21 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:31:47.998169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:31:47.998807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:31:48.011210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:31:48.011210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:31:48.011210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:31:48.011210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:31:48.011741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:31:48.019823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1084/2156] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:31:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:31:58.016004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:32:00.250055 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:32:00.251723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:32:00.264108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44600e0a-990e-4dac-b18c-3fa9a5600cde service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:32:00.267696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:32:00.268341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:32:00.268627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:32:00.268906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:32:00.270729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:32:00.329798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:32:00.331347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:32:00.340877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1073/2157] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:32:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:32:10.336322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=28 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:32:39.170735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:32:39.180111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536f32b-2fa3-4ea8-8eab-259f4e535674 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:32:39.184341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c370780-eb73-4d8d-8a4c-d622e598b03a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:32:39.185011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c370780-eb73-4d8d-8a4c-d622e598b03a service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:32:39.185293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c370780-eb73-4d8d-8a4c-d622e598b03a service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:32:39.185624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c370780-eb73-4d8d-8a4c-d622e598b03a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:32:39.187419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1c370780-eb73-4d8d-8a4c-d622e598b03a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:32:39.198884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c370780-eb73-4d8d-8a4c-d622e598b03a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:32:39.200319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c370780-eb73-4d8d-8a4c-d622e598b03a service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:32:39.209461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1085/2158] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:32:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:32:49.205095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:34:56.273206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:34:56.344925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9899db1-951d-4f7e-acad-f2e20c46be0c service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:34:56.350091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4ab2e-2925-40ae-b3dd-e7829e265434 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:34:56.350732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4ab2e-2925-40ae-b3dd-e7829e265434 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:34:56.351013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4ab2e-2925-40ae-b3dd-e7829e265434 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:34:56.351309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4ab2e-2925-40ae-b3dd-e7829e265434 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:34:56.359901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4ab2e-2925-40ae-b3dd-e7829e265434 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:34:56.361460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4ab2e-2925-40ae-b3dd-e7829e265434 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:34:56.371010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1074/2159] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:34:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:35:06.366903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=27 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:35:19.589993 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c138-7304-49e7-a2ea-88712f366cfc None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:19.590932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c138-7304-49e7-a2ea-88712f366cfc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:19.591399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c138-7304-49e7-a2ea-88712f366cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:19.630351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e9f7c138-7304-49e7-a2ea-88712f366cfc None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:35:19.631651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e9f7c138-7304-49e7-a2ea-88712f366cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:19.646298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1086/2160] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 11 11:35:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:35:19.825217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4ab2e-2925-40ae-b3dd-e7829e265434 service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:35:19.836092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4ab2e-2925-40ae-b3dd-e7829e265434 service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:35:19.840976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e36440-5f36-459c-b34b-8cfcccdc2ddb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:35:19.841604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36440-5f36-459c-b34b-8cfcccdc2ddb service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:19.841884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36440-5f36-459c-b34b-8cfcccdc2ddb service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:19.842156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e36440-5f36-459c-b34b-8cfcccdc2ddb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:19.850786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e36440-5f36-459c-b34b-8cfcccdc2ddb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:35:19.852374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e36440-5f36-459c-b34b-8cfcccdc2ddb service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:35:19.860493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1075/2161] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 11:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:35:21.823552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c138-7304-49e7-a2ea-88712f366cfc None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:35:21.824958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e9f7c138-7304-49e7-a2ea-88712f366cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:21.862424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c138-7304-49e7-a2ea-88712f366cfc None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:35:21.868596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af1326-034a-4d8d-9869-178ede0066c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:35:21.869395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af1326-034a-4d8d-9869-178ede0066c3 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:21.869800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af1326-034a-4d8d-9869-178ede0066c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:21.870150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af1326-034a-4d8d-9869-178ede0066c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:21.872165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d3af1326-034a-4d8d-9869-178ede0066c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:21.891272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af1326-034a-4d8d-9869-178ede0066c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:35:21.894673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af1326-034a-4d8d-9869-178ede0066c3 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:35:21.906890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1087/2162] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:35:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:35:22.079668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8bb7e-4afe-4812-9b69-ad7a9baeab82 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:22.079668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8bb7e-4afe-4812-9b69-ad7a9baeab82 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:22.079668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8bb7e-4afe-4812-9b69-ad7a9baeab82 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:22.080346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1076/2163] 10.222.0.26 () {54 vars in 912 bytes} [Mon Jul 11 11:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:35:22.085659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e08cdb-b9ee-419a-aa64-b1e01f105c86 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:22.086024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e08cdb-b9ee-419a-aa64-b1e01f105c86 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:22.086418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e08cdb-b9ee-419a-aa64-b1e01f105c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:22.167859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a1e08cdb-b9ee-419a-aa64-b1e01f105c86 None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:35:22.169143 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a1e08cdb-b9ee-419a-aa64-b1e01f105c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:22.180827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1088/2164] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:35:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:35:22.204454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8bb7e-4afe-4812-9b69-ad7a9baeab82 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:35:22.218301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8bb7e-4afe-4812-9b69-ad7a9baeab82 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:35:22.223858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b69ca-4ee5-40d9-9489-3b809f1a93d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:35:22.224499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b69ca-4ee5-40d9-9489-3b809f1a93d0 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:22.224783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b69ca-4ee5-40d9-9489-3b809f1a93d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:22.225060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b69ca-4ee5-40d9-9489-3b809f1a93d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:22.233551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b69ca-4ee5-40d9-9489-3b809f1a93d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:35:22.235131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b69ca-4ee5-40d9-9489-3b809f1a93d0 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:35:22.245838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1077/2165] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:35:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:35:26.389952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3d0e8-e3cd-4b6d-ad6c-5b3e16a2a8dc None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:26.390491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3d0e8-e3cd-4b6d-ad6c-5b3e16a2a8dc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:26.390678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3d0e8-e3cd-4b6d-ad6c-5b3e16a2a8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:26.411725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3ec3d0e8-e3cd-4b6d-ad6c-5b3e16a2a8dc None None] MFA Rules not processed for user `993a59e8746948ab8a0f11d877314e02`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:35:26.413763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3ec3d0e8-e3cd-4b6d-ad6c-5b3e16a2a8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:26.423115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1089/2166] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:35:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:35:26.722072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b69ca-4ee5-40d9-9489-3b809f1a93d0 service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:35:26.741639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b69ca-4ee5-40d9-9489-3b809f1a93d0 service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:35:26.745146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:35:26.745785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:26.746064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:26.746337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:26.748117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:26.770392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:35:26.771502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:35:26.779558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1078/2167] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:35:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:35:27.190926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3d0e8-e3cd-4b6d-ad6c-5b3e16a2a8dc None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:35:27.193419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3ec3d0e8-e3cd-4b6d-ad6c-5b3e16a2a8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:27.208716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3d0e8-e3cd-4b6d-ad6c-5b3e16a2a8dc None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:35:27.213835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e63338-e52c-4202-909f-60088df1219f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:35:27.214418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63338-e52c-4202-909f-60088df1219f service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:27.214700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63338-e52c-4202-909f-60088df1219f service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:27.214974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e63338-e52c-4202-909f-60088df1219f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:27.216729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-28e63338-e52c-4202-909f-60088df1219f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:27.228268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e63338-e52c-4202-909f-60088df1219f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:35:27.229876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e63338-e52c-4202-909f-60088df1219f service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:35:27.244639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1090/2168] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:35:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:35:31.023298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:35:31.042329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8af60-8830-46b0-8de3-0cbb5e64a9c5 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:35:31.055026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:35:31.056105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:31.056592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:31.057093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:31.059468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:31.074012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:35:31.075983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:35:31.094339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1079/2169] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:35:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:35:37.138966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e63338-e52c-4202-909f-60088df1219f service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:35:37.151835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e63338-e52c-4202-909f-60088df1219f service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:35:37.157182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:35:37.159202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:35:37.159202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:35:37.159202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:35:37.160263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:35:37.172599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:35:37.174176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:35:37.184654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1091/2170] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:35:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:35:41.089337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=33 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:35:47.180251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=62 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:36:13.605692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:36:13.618150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d3596-1033-4185-bce2-df341a202d4d service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:36:13.623706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c215e-292d-414f-aabe-e97a960e610a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:36:13.624344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c215e-292d-414f-aabe-e97a960e610a service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:36:13.624631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c215e-292d-414f-aabe-e97a960e610a service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:36:13.624911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c215e-292d-414f-aabe-e97a960e610a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:36:13.632735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c215e-292d-414f-aabe-e97a960e610a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:36:13.634310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c215e-292d-414f-aabe-e97a960e610a service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:36:13.661849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1080/2171] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:36:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:36:13.668957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:36:13.681675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac586a4-8bf9-47bf-9e56-215a66faa8ce service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:36:13.690857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed401a-1fce-49f0-bec4-df2fc9dfb1b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:36:13.691661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed401a-1fce-49f0-bec4-df2fc9dfb1b0 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:36:13.691952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed401a-1fce-49f0-bec4-df2fc9dfb1b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:36:13.692226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed401a-1fce-49f0-bec4-df2fc9dfb1b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:36:13.701521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed401a-1fce-49f0-bec4-df2fc9dfb1b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:36:13.703080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed401a-1fce-49f0-bec4-df2fc9dfb1b0 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:36:13.713065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c215e-292d-414f-aabe-e97a960e610a service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:36:13.723803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c215e-292d-414f-aabe-e97a960e610a service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:36:13.729683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0dbc7-119d-4bf5-b0c2-fe8d2506cdf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:36:13.731618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0dbc7-119d-4bf5-b0c2-fe8d2506cdf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:36:13.731618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0dbc7-119d-4bf5-b0c2-fe8d2506cdf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:36:13.731618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0dbc7-119d-4bf5-b0c2-fe8d2506cdf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:36:13.740084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1092/2172] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:36:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:36:13.746679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0dbc7-119d-4bf5-b0c2-fe8d2506cdf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:36:13.748078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0dbc7-119d-4bf5-b0c2-fe8d2506cdf6 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:36:13.760573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1081/2173] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:36:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:36:13.794139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed401a-1fce-49f0-bec4-df2fc9dfb1b0 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:36:13.805402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed401a-1fce-49f0-bec4-df2fc9dfb1b0 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:36:13.811379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7a1ea-9bcf-4cdc-a75d-c322cdf5609f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:36:13.812145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a1ea-9bcf-4cdc-a75d-c322cdf5609f service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:36:13.812435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a1ea-9bcf-4cdc-a75d-c322cdf5609f service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:36:13.812713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7a1ea-9bcf-4cdc-a75d-c322cdf5609f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:36:13.822448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7a1ea-9bcf-4cdc-a75d-c322cdf5609f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:36:13.824164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7a1ea-9bcf-4cdc-a75d-c322cdf5609f service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:36:13.852046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1093/2174] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:36:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:36:14.935712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0dbc7-119d-4bf5-b0c2-fe8d2506cdf6 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:36:14.944602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0dbc7-119d-4bf5-b0c2-fe8d2506cdf6 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:36:14.948293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d86faf-6224-44c8-af06-3c58958d48fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:36:14.949069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86faf-6224-44c8-af06-3c58958d48fe service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:36:14.949186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86faf-6224-44c8-af06-3c58958d48fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:36:14.949653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86faf-6224-44c8-af06-3c58958d48fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:36:14.960591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d86faf-6224-44c8-af06-3c58958d48fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:36:14.962004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d86faf-6224-44c8-af06-3c58958d48fe service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:36:14.972867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1082/2175] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:36:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:36:15.366827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7a1ea-9bcf-4cdc-a75d-c322cdf5609f service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:36:15.377855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7a1ea-9bcf-4cdc-a75d-c322cdf5609f service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:36:15.383792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c64e0-5c14-4d56-85e5-5ae06f6b0199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:36:15.384447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c64e0-5c14-4d56-85e5-5ae06f6b0199 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:36:15.384723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c64e0-5c14-4d56-85e5-5ae06f6b0199 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:36:15.385007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c64e0-5c14-4d56-85e5-5ae06f6b0199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:36:15.392768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c64e0-5c14-4d56-85e5-5ae06f6b0199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:36:15.394341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c64e0-5c14-4d56-85e5-5ae06f6b0199 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:36:15.427453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1094/2176] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:36:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:36:24.967917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=25 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:36:25.425247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=39 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:00.290667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d86faf-6224-44c8-af06-3c58958d48fe service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:00.309423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d86faf-6224-44c8-af06-3c58958d48fe service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:00.312979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa3e04-ec02-4040-b939-6d6f8e73a68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:00.313614 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa3e04-ec02-4040-b939-6d6f8e73a68f service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:00.313899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa3e04-ec02-4040-b939-6d6f8e73a68f service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:00.314176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa3e04-ec02-4040-b939-6d6f8e73a68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:00.323132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa3e04-ec02-4040-b939-6d6f8e73a68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:00.324621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa3e04-ec02-4040-b939-6d6f8e73a68f service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:00.335338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1083/2177] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:37:10.329507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=10 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:16.573873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c64e0-5c14-4d56-85e5-5ae06f6b0199 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:16.584517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c64e0-5c14-4d56-85e5-5ae06f6b0199 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:16.589006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac07af8-eb83-43ca-bf37-a80901aaa678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:16.589664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac07af8-eb83-43ca-bf37-a80901aaa678 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:16.589951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac07af8-eb83-43ca-bf37-a80901aaa678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:16.590230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac07af8-eb83-43ca-bf37-a80901aaa678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:16.592135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4ac07af8-eb83-43ca-bf37-a80901aaa678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:16.700724 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac07af8-eb83-43ca-bf37-a80901aaa678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:16.702268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac07af8-eb83-43ca-bf37-a80901aaa678 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:16.721751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1095/2178] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:37:17.140352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa3e04-ec02-4040-b939-6d6f8e73a68f service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:17.148779 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa3e04-ec02-4040-b939-6d6f8e73a68f service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:17.153059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a16e9-fdb4-46e6-9080-cedbf4a5e1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:17.153929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a16e9-fdb4-46e6-9080-cedbf4a5e1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:17.154425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a16e9-fdb4-46e6-9080-cedbf4a5e1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:17.154868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a16e9-fdb4-46e6-9080-cedbf4a5e1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:17.166731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a16e9-fdb4-46e6-9080-cedbf4a5e1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:17.168669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a16e9-fdb4-46e6-9080-cedbf4a5e1a9 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:17.178332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1084/2179] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:37:17.366200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0f5a-e0c2-468f-b52c-445b0ad679da None None] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:17.366200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0f5a-e0c2-468f-b52c-445b0ad679da None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:17.366200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0f5a-e0c2-468f-b52c-445b0ad679da None None] PATH_INFO: `/` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:17.368816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1096/2180] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 11:37:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 11:37:17.389995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1f5e-4ea7-4b6b-a9ee-00314058a45d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:17.390363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1f5e-4ea7-4b6b-a9ee-00314058a45d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:17.390709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1f5e-4ea7-4b6b-a9ee-00314058a45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:17.409434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-715c1f5e-4ea7-4b6b-a9ee-00314058a45d None None] MFA Rules not processed for user `94f2124d6d4e46ca9baa9728741b0e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:17.410981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-715c1f5e-4ea7-4b6b-a9ee-00314058a45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:17.421182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1085/2181] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 11:37:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:17.447558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf0f5a-e0c2-468f-b52c-445b0ad679da None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:17.462533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf0f5a-e0c2-468f-b52c-445b0ad679da None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:17.467841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da2de8-d4de-4af7-a1e8-9a4b3f19efd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:17.468804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da2de8-d4de-4af7-a1e8-9a4b3f19efd8 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:17.469189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da2de8-d4de-4af7-a1e8-9a4b3f19efd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:17.469567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da2de8-d4de-4af7-a1e8-9a4b3f19efd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:17.481293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da2de8-d4de-4af7-a1e8-9a4b3f19efd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:17.484053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da2de8-d4de-4af7-a1e8-9a4b3f19efd8 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:17.496656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1097/2182] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:37:17.602855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c1f5e-4ea7-4b6b-a9ee-00314058a45d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:17.605079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-715c1f5e-4ea7-4b6b-a9ee-00314058a45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:17.626578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c1f5e-4ea7-4b6b-a9ee-00314058a45d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:17.631714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:17.633430 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:17.633711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:17.634103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] PATH_INFO: `/v3/users/9dc29ac59eed483b8c6443c397505adc` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:17.635271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc29ac59eed483b8c6443c397505adc)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:17.637205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:17.852290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc29ac59eed483b8c6443c397505adc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:17.853471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc29ac59eed483b8c6443c397505adc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:17.854776 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc29ac59eed483b8c6443c397505adc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:17.878822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc29ac59eed483b8c6443c397505adc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:17.895568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-46e688a9-c554-414a-b1b6-cfd27c137ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc29ac59eed483b8c6443c397505adc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:17.993826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1086/2183] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:37:17 2022] DELETE /identity/v3/users/9dc29ac59eed483b8c6443c397505adc => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:37:18.297141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da2de8-d4de-4af7-a1e8-9a4b3f19efd8 service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:18.312221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da2de8-d4de-4af7-a1e8-9a4b3f19efd8 service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:18.381543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:18.382468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:18.382758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:18.383035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] PATH_INFO: `/v3/projects/890e7e79ff744b6b9589f166cd7d8979` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:18.384285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=890e7e79ff744b6b9589f166cd7d8979)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:18.386193 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:18.551375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890e7e79ff744b6b9589f166cd7d8979', 'request_id': 'req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:18.555344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:18.557633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890e7e79ff744b6b9589f166cd7d8979', 'request_id': 'req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:18.720897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890e7e79ff744b6b9589f166cd7d8979', 'request_id': 'req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:18.732380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] The token cache is being invalidate because project 890e7e79ff744b6b9589f166cd7d8979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:37:18.733574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:18.733683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6977ba5a-4c02-4c37-88e8-6f0acce12b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:18.735136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1098/2184] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:37:18 2022] DELETE /identity/v3/projects/890e7e79ff744b6b9589f166cd7d8979 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:37:18.742716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aa3d1-da19-4e43-8c84-eb27b973226a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:18.743233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aa3d1-da19-4e43-8c84-eb27b973226a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:18.743391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5aa3d1-da19-4e43-8c84-eb27b973226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:18.792728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5f5aa3d1-da19-4e43-8c84-eb27b973226a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:18.793928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5f5aa3d1-da19-4e43-8c84-eb27b973226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:18.851182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1087/2185] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:18.859338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5115be3-b18a-4fcf-b1ae-23f9802e95a8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:18.859633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5115be3-b18a-4fcf-b1ae-23f9802e95a8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:18.859943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5115be3-b18a-4fcf-b1ae-23f9802e95a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:18.878603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b5115be3-b18a-4fcf-b1ae-23f9802e95a8 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:18.879831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b5115be3-b18a-4fcf-b1ae-23f9802e95a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:18.888747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1099/2186] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:18.893024 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5aa3d1-da19-4e43-8c84-eb27b973226a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:18.903339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5aa3d1-da19-4e43-8c84-eb27b973226a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:18.907820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:18.908470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:18.908751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:18.909026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:18.909789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:37:18.910101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:18.911722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:18.923327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1088/2187] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:37:18.927686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5115be3-b18a-4fcf-b1ae-23f9802e95a8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:18.937684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5115be3-b18a-4fcf-b1ae-23f9802e95a8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:18.942333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4e604-d295-4f5c-9855-215a104b681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:18.942982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4e604-d295-4f5c-9855-215a104b681e admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:18.943289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4e604-d295-4f5c-9855-215a104b681e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:18.943572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4e604-d295-4f5c-9855-215a104b681e admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:18.944528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4e604-d295-4f5c-9855-215a104b681e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:18.947083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4e604-d295-4f5c-9855-215a104b681e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:19.074314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1100/2188] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:37:18 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:37:19.078030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:19.088004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175aba0f-d319-4db9-bf22-2c46005ee59d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:19.100601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:19.143663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bcddc-0f64-4f97-969d-0a5105837715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:19.144261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcddc-0f64-4f97-969d-0a5105837715 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:19.144550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcddc-0f64-4f97-969d-0a5105837715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:19.144828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0bcddc-0f64-4f97-969d-0a5105837715 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:19.145515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bcddc-0f64-4f97-969d-0a5105837715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:19.147467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0bcddc-0f64-4f97-969d-0a5105837715 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:19.269576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1089/2189] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:37:19 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:37:19.273600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4e604-d295-4f5c-9855-215a104b681e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:19.284308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4e604-d295-4f5c-9855-215a104b681e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:19.294702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54da613-b7e1-4d2d-9af9-dd3e5d8ff09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:19.294702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54da613-b7e1-4d2d-9af9-dd3e5d8ff09d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:19.294702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54da613-b7e1-4d2d-9af9-dd3e5d8ff09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:19.294702 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54da613-b7e1-4d2d-9af9-dd3e5d8ff09d admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:19.295320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54da613-b7e1-4d2d-9af9-dd3e5d8ff09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:19.295320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54da613-b7e1-4d2d-9af9-dd3e5d8ff09d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:19.303810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1101/2190] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:37:19.308439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bcddc-0f64-4f97-969d-0a5105837715 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:19.318357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0bcddc-0f64-4f97-969d-0a5105837715 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:19.323532 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377de46c-b228-4031-857c-94668b63dab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:19.324195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377de46c-b228-4031-857c-94668b63dab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:19.324477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377de46c-b228-4031-857c-94668b63dab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:19.324750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377de46c-b228-4031-857c-94668b63dab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:19.325588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377de46c-b228-4031-857c-94668b63dab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:19.327242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377de46c-b228-4031-857c-94668b63dab4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:19.337464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1090/2191] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:37:19.341457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54da613-b7e1-4d2d-9af9-dd3e5d8ff09d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:19.352615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54da613-b7e1-4d2d-9af9-dd3e5d8ff09d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:19.357271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:19.358703 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:19.358997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:19.359308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] PATH_INFO: `/v3/projects/c88b5670c8394b3c97bc250eaa23b152/users/d9bd75268bb1436d92c4d74f2cfa6944/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:19.374793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88b5670c8394b3c97bc250eaa23b152, user_id=d9bd75268bb1436d92c4d74f2cfa6944, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:19.377488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:19.464159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1102/2192] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:37:19 2022] PUT /identity/v3/projects/c88b5670c8394b3c97bc250eaa23b152/users/d9bd75268bb1436d92c4d74f2cfa6944/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:37:19.474188 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633e08-f34b-4190-9992-960fd61bf672 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:19.474484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633e08-f34b-4190-9992-960fd61bf672 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:19.474839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9633e08-f34b-4190-9992-960fd61bf672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:19.502092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f9633e08-f34b-4190-9992-960fd61bf672 None None] MFA Rules not processed for user `d9bd75268bb1436d92c4d74f2cfa6944`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:19.503327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f9633e08-f34b-4190-9992-960fd61bf672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:19.566524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1091/2193] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:19.578007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:19.579725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:19.646620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d59d4-de6a-4348-9d0a-692d8b2b3212 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:19.650736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719c713-6662-4fd9-a1fb-c664bbd34ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:19.651414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719c713-6662-4fd9-a1fb-c664bbd34ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:19.651699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719c713-6662-4fd9-a1fb-c664bbd34ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:19.652000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719c713-6662-4fd9-a1fb-c664bbd34ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:19.660121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719c713-6662-4fd9-a1fb-c664bbd34ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:19.661539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719c713-6662-4fd9-a1fb-c664bbd34ef2 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:19.670945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1103/2194] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:37:19.694448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9633e08-f34b-4190-9992-960fd61bf672 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:19.760568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9633e08-f34b-4190-9992-960fd61bf672 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:19.765143 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:19.766008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:19.766289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:19.766564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] PATH_INFO: `/v3/users/d9bd75268bb1436d92c4d74f2cfa6944` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:19.767785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bd75268bb1436d92c4d74f2cfa6944)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:19.769716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:19.896818 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:20.002838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bd75268bb1436d92c4d74f2cfa6944'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.003508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bd75268bb1436d92c4d74f2cfa6944'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.004132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bd75268bb1436d92c4d74f2cfa6944'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.011629 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bd75268bb1436d92c4d74f2cfa6944'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.029535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bd75268bb1436d92c4d74f2cfa6944'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.106060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1092/2195] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:37:19 2022] DELETE /identity/v3/users/d9bd75268bb1436d92c4d74f2cfa6944 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:37:20.110586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599493a6-cd5e-444a-99ce-05d983b47d00 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:20.110942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599493a6-cd5e-444a-99ce-05d983b47d00 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:20.111224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599493a6-cd5e-444a-99ce-05d983b47d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:20.128369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-599493a6-cd5e-444a-99ce-05d983b47d00 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:20.129547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-599493a6-cd5e-444a-99ce-05d983b47d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:20.186934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1104/2196] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:20.197213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:20.198659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:20.254995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3e3a0-e985-4d8c-bcd0-79a5f5044bd1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:20.259356 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638242b-272f-4a7d-9a67-11881936db01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:20.260012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638242b-272f-4a7d-9a67-11881936db01 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:20.260316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638242b-272f-4a7d-9a67-11881936db01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:20.260575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638242b-272f-4a7d-9a67-11881936db01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:20.270058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638242b-272f-4a7d-9a67-11881936db01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:20.271535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638242b-272f-4a7d-9a67-11881936db01 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:20.283209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1093/2197] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:37:20.667387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599493a6-cd5e-444a-99ce-05d983b47d00 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:20.668889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-599493a6-cd5e-444a-99ce-05d983b47d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:20.677458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:20.682487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599493a6-cd5e-444a-99ce-05d983b47d00 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:20.687137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:20.688011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:20.688290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:20.688566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] PATH_INFO: `/v3/projects/c88b5670c8394b3c97bc250eaa23b152` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:20.689734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88b5670c8394b3c97bc250eaa23b152)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:20.691805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:20.792624 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88b5670c8394b3c97bc250eaa23b152', 'request_id': 'req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.797924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88b5670c8394b3c97bc250eaa23b152', 'request_id': 'req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.863149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88b5670c8394b3c97bc250eaa23b152', 'request_id': 'req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.875177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] The token cache is being invalidate because project c88b5670c8394b3c97bc250eaa23b152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:37:20.875502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.876131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7ae59e59-89e4-41e5-8dc4-57ddaec710d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:37:20.877653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1105/2198] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:37:20 2022] DELETE /identity/v3/projects/c88b5670c8394b3c97bc250eaa23b152 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:37:20.882665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07fc4-594f-4d44-a471-80475f2a337c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:20.883130 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07fc4-594f-4d44-a471-80475f2a337c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:20.883274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf07fc4-594f-4d44-a471-80475f2a337c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:20.900449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0bf07fc4-594f-4d44-a471-80475f2a337c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:20.901642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0bf07fc4-594f-4d44-a471-80475f2a337c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:20.958996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1094/2199] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:20.966621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5dc76-4f57-42f5-9b93-582c9cbd2320 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:20.966969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5dc76-4f57-42f5-9b93-582c9cbd2320 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:20.967273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5dc76-4f57-42f5-9b93-582c9cbd2320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:20.985512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-15c5dc76-4f57-42f5-9b93-582c9cbd2320 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:20.986741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-15c5dc76-4f57-42f5-9b93-582c9cbd2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:20.995300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1106/2200] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:20.999278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf07fc4-594f-4d44-a471-80475f2a337c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:21.008878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf07fc4-594f-4d44-a471-80475f2a337c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:21.013529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:21.014182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:21.014469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:21.014748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:21.015508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:37:21.015822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:21.017436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:21.028797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1095/2201] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:37:21.032984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5dc76-4f57-42f5-9b93-582c9cbd2320 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:21.042762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5dc76-4f57-42f5-9b93-582c9cbd2320 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:21.047359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260712d-8db0-4ec8-8f0f-54f133ca72cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:21.048039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260712d-8db0-4ec8-8f0f-54f133ca72cb admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:21.048326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260712d-8db0-4ec8-8f0f-54f133ca72cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:21.048607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260712d-8db0-4ec8-8f0f-54f133ca72cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:21.049310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260712d-8db0-4ec8-8f0f-54f133ca72cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:21.051224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260712d-8db0-4ec8-8f0f-54f133ca72cb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:21.136457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1107/2202] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:37:21 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:37:21.140460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:21.151043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524f8e-1ec2-45d9-a494-70e645d73a16 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:21.182892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:21.206139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f06af-3467-46c6-b609-2312e634f2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:21.206797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f06af-3467-46c6-b609-2312e634f2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:21.207075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f06af-3467-46c6-b609-2312e634f2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:21.207377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f06af-3467-46c6-b609-2312e634f2f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:21.208151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f06af-3467-46c6-b609-2312e634f2f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:21.210051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f06af-3467-46c6-b609-2312e634f2f9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:21.297756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1096/2203] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:37:21 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:37:21.303990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260712d-8db0-4ec8-8f0f-54f133ca72cb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:21.315160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260712d-8db0-4ec8-8f0f-54f133ca72cb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:21.320434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef01ef2-5153-4ddd-9cbc-c7f1a56d5aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:21.321376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef01ef2-5153-4ddd-9cbc-c7f1a56d5aab admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:21.322102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef01ef2-5153-4ddd-9cbc-c7f1a56d5aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:21.322458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef01ef2-5153-4ddd-9cbc-c7f1a56d5aab admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:21.323571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef01ef2-5153-4ddd-9cbc-c7f1a56d5aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:21.325273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef01ef2-5153-4ddd-9cbc-c7f1a56d5aab admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:21.339838 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1108/2204] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:37:21.340907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f06af-3467-46c6-b609-2312e634f2f9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:21.351127 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f06af-3467-46c6-b609-2312e634f2f9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:21.357156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e96fc7-ad0c-4983-bcf2-f54351aa331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:21.357916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96fc7-ad0c-4983-bcf2-f54351aa331d admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:21.358327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96fc7-ad0c-4983-bcf2-f54351aa331d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:21.358716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96fc7-ad0c-4983-bcf2-f54351aa331d admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:21.359637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e96fc7-ad0c-4983-bcf2-f54351aa331d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:21.361337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e96fc7-ad0c-4983-bcf2-f54351aa331d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:21.375207 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1097/2205] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:37:21.376617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef01ef2-5153-4ddd-9cbc-c7f1a56d5aab admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:21.386984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef01ef2-5153-4ddd-9cbc-c7f1a56d5aab admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:21.393540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:21.394433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:21.394716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:21.394994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] PATH_INFO: `/v3/projects/72a461d4496f4015969f2683af43afa7/users/559a564c171f4dd1abb3e4af27987bbd/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:21.414905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a461d4496f4015969f2683af43afa7, user_id=559a564c171f4dd1abb3e4af27987bbd, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:21.417622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:21.509987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1109/2206] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:37:21 2022] PUT /identity/v3/projects/72a461d4496f4015969f2683af43afa7/users/559a564c171f4dd1abb3e4af27987bbd/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:37:21.515186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5283f71-0796-4147-97f3-d752158fb970 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:21.515571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5283f71-0796-4147-97f3-d752158fb970 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:21.515917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5283f71-0796-4147-97f3-d752158fb970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:21.534524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-d5283f71-0796-4147-97f3-d752158fb970 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:21.535733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d5283f71-0796-4147-97f3-d752158fb970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:21.595529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1098/2207] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:21.605100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:21.606800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:21.662756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0c329-5804-4c9d-99e4-3cd07afc6581 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:21.666454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c726a5-e2cf-46d4-b42d-0d939b3a7e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:21.667101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c726a5-e2cf-46d4-b42d-0d939b3a7e8c service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:21.667430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c726a5-e2cf-46d4-b42d-0d939b3a7e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:21.667978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c726a5-e2cf-46d4-b42d-0d939b3a7e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:21.676738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c726a5-e2cf-46d4-b42d-0d939b3a7e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:21.678168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c726a5-e2cf-46d4-b42d-0d939b3a7e8c service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:21.687563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1110/2208] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:37:27.090661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04cc34-9506-4fa5-9639-6c02e976f8fc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:27.091181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04cc34-9506-4fa5-9639-6c02e976f8fc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:27.091339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04cc34-9506-4fa5-9639-6c02e976f8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:27.120835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6a04cc34-9506-4fa5-9639-6c02e976f8fc None None] MFA Rules not processed for user `559a564c171f4dd1abb3e4af27987bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:27.122301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6a04cc34-9506-4fa5-9639-6c02e976f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:27.213087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1099/2209] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:27.217941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c726a5-e2cf-46d4-b42d-0d939b3a7e8c service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:27.228627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c726a5-e2cf-46d4-b42d-0d939b3a7e8c service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:27.233307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f528b-ba82-4f56-a7f8-5d9ee53718ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:27.234184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f528b-ba82-4f56-a7f8-5d9ee53718ef admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:27.234634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f528b-ba82-4f56-a7f8-5d9ee53718ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:27.234912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f528b-ba82-4f56-a7f8-5d9ee53718ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:27.235824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f528b-ba82-4f56-a7f8-5d9ee53718ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:27.241345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f528b-ba82-4f56-a7f8-5d9ee53718ef admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:27.377050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1111/2210] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:37:27 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:37:27.381047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04cc34-9506-4fa5-9639-6c02e976f8fc None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:27.392159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04cc34-9506-4fa5-9639-6c02e976f8fc None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:27.435880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:27.451330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c6fa3-ef0b-4e1a-896f-61adcf84d02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:27.452041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c6fa3-ef0b-4e1a-896f-61adcf84d02b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:27.452324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c6fa3-ef0b-4e1a-896f-61adcf84d02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:27.452600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c6fa3-ef0b-4e1a-896f-61adcf84d02b admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:27.453307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c6fa3-ef0b-4e1a-896f-61adcf84d02b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:27.455242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c6fa3-ef0b-4e1a-896f-61adcf84d02b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:27.584118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1100/2211] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:37:27 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:37:27.588783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f528b-ba82-4f56-a7f8-5d9ee53718ef admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:27.594231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:27.599488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f528b-ba82-4f56-a7f8-5d9ee53718ef admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:27.604038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421adf8-3f16-4443-9718-315865fbec10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:27.604693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421adf8-3f16-4443-9718-315865fbec10 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:27.604977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421adf8-3f16-4443-9718-315865fbec10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:27.605254 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421adf8-3f16-4443-9718-315865fbec10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:27.605973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421adf8-3f16-4443-9718-315865fbec10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:27.607716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421adf8-3f16-4443-9718-315865fbec10 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:27.618052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1112/2212] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:37:27.628387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c6fa3-ef0b-4e1a-896f-61adcf84d02b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:27.638948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c6fa3-ef0b-4e1a-896f-61adcf84d02b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:27.644287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e812935-a874-4e6c-8ce6-ceb0475a8de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:27.645067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812935-a874-4e6c-8ce6-ceb0475a8de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:27.645350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812935-a874-4e6c-8ce6-ceb0475a8de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:27.645628 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812935-a874-4e6c-8ce6-ceb0475a8de1 admin admin] PATH_INFO: `/v3/projects/0b4f15a303e245d4ab443dcd83e82bfd/users/cea34f32015d4694896f3266e2218d93/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:27.662225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e812935-a874-4e6c-8ce6-ceb0475a8de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4f15a303e245d4ab443dcd83e82bfd, user_id=cea34f32015d4694896f3266e2218d93, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:27.665155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e812935-a874-4e6c-8ce6-ceb0475a8de1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:27.766189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1101/2213] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:37:27 2022] PUT /identity/v3/projects/0b4f15a303e245d4ab443dcd83e82bfd/users/cea34f32015d4694896f3266e2218d93/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:37:27.775006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421adf8-3f16-4443-9718-315865fbec10 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:27.784935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421adf8-3f16-4443-9718-315865fbec10 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:27.843632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea2b37-e254-45c9-aff3-11a4c5dcfa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:27.844309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea2b37-e254-45c9-aff3-11a4c5dcfa2a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:27.844602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea2b37-e254-45c9-aff3-11a4c5dcfa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:27.844885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea2b37-e254-45c9-aff3-11a4c5dcfa2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:27.845611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea2b37-e254-45c9-aff3-11a4c5dcfa2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:27.847238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea2b37-e254-45c9-aff3-11a4c5dcfa2a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:27.858888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1113/2214] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:37:27.862693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e812935-a874-4e6c-8ce6-ceb0475a8de1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:27.873783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e812935-a874-4e6c-8ce6-ceb0475a8de1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:27.878711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:27.879763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:27.880049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:27.880327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] PATH_INFO: `/v3/projects/0b4f15a303e245d4ab443dcd83e82bfd/users/cea34f32015d4694896f3266e2218d93/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:27.882315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4f15a303e245d4ab443dcd83e82bfd, user_id=cea34f32015d4694896f3266e2218d93, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:27.885285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:27.897079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1102/2215] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:37:27 2022] PUT /identity/v3/projects/0b4f15a303e245d4ab443dcd83e82bfd/users/cea34f32015d4694896f3266e2218d93/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:37:33.650630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b05719-6597-4cce-9e6e-b4b4f685b9d9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:33.651166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b05719-6597-4cce-9e6e-b4b4f685b9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:33.651304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b05719-6597-4cce-9e6e-b4b4f685b9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:33.680418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-28b05719-6597-4cce-9e6e-b4b4f685b9d9 None None] MFA Rules not processed for user `cea34f32015d4694896f3266e2218d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:33.681598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-28b05719-6597-4cce-9e6e-b4b4f685b9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:33.755056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1114/2216] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:33.768181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:33.769895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:33.873949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9c93-8589-458d-9302-eff8893c017c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:33.874239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e939bc73c40140cb89517e918722d154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:33.874898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:33.875181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:33.875677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:33.935558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:33.937021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:33.946772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1103/2217] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:37:34.256867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbb87-5555-4afb-8a73-1723132188fe None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:34.257207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbb87-5555-4afb-8a73-1723132188fe None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:34.257522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdbb87-5555-4afb-8a73-1723132188fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:34.317899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f6cdbb87-5555-4afb-8a73-1723132188fe None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:34.319150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f6cdbb87-5555-4afb-8a73-1723132188fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:34.363511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:34.385171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1115/2218] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:37:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:34.533548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:34.534950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:34.569621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:34.628250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15db19a-af83-478a-9242-741f58099e1a service glance] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:34.634050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:34.634706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:34.634992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:34.635303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:34.643520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:34.645103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:34.653087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1104/2219] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 11:37:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:37:44.409354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:44.649871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=27 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:45.369321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334035a6-78a5-44ef-922e-6773a6bbd547 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:45.369628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334035a6-78a5-44ef-922e-6773a6bbd547 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:45.369916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334035a6-78a5-44ef-922e-6773a6bbd547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:45.386926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-334035a6-78a5-44ef-922e-6773a6bbd547 None None] MFA Rules not processed for user `559a564c171f4dd1abb3e4af27987bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:45.388160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-334035a6-78a5-44ef-922e-6773a6bbd547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:45.394939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1116/2220] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:37:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:45.660925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:45.662360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:45.717488 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fbc79-352d-4788-801f-a4c48679007e service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:45.721090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:45.721726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:45.722015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:45.722291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:45.724060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:45.733693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:45.735109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:45.742907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1105/2221] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:37:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:37:49.011884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcd766-0285-42c1-8f1c-3e74b172eaa8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:49.012454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcd766-0285-42c1-8f1c-3e74b172eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:49.012454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcd766-0285-42c1-8f1c-3e74b172eaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:49.031207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8dbcd766-0285-42c1-8f1c-3e74b172eaa8 None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:37:49.032524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8dbcd766-0285-42c1-8f1c-3e74b172eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:37:49.039784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1117/2222] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 11:37:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:37:49.052720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:37:49.066655 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfbb22-aeae-4acb-9c28-be1169328f37 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:37:49.071218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:37:49.071899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:37:49.072185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:37:49.072463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:37:49.081268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:37:49.082832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:37:49.091597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1106/2223] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 11:37:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:37:59.025699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:37:59.088153 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=32 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:40:01.284597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcd766-0285-42c1-8f1c-3e74b172eaa8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:40:01.286605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8dbcd766-0285-42c1-8f1c-3e74b172eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:40:01.355854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcd766-0285-42c1-8f1c-3e74b172eaa8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:40:01.360843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:40:01.361521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:40:01.361996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:40:01.362112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:40:01.364462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:40:01.376411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:40:01.377990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:40:01.388066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1118/2224] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:40:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:40:11.383890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=32 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:41:13.842697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:41:13.844108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:41:13.898635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc035ac-e109-4914-8bfe-c7dd258337bc service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:41:13.902138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:41:13.902770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:41:13.903052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:41:13.903360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:41:13.905078 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:41:13.973754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:41:13.975172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:41:13.984012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1107/2225] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:41:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:41:14.031033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:41:14.041140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0735a6-9366-48a4-a2ef-852f3365489c service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:41:14.045004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:41:14.045661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:41:14.045948 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:41:14.046230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:41:14.048028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:41:14.118583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:41:14.120058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:41:14.130191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1119/2226] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:41:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:41:14.218552 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:41:14.227847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc6cdf-97e6-4e22-a9b9-7770004db1dc service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:41:14.232054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:41:14.232959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:41:14.233647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:41:14.233936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:41:14.236249 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:41:14.253759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:41:14.262138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd8017-03d6-440c-8fdf-ff97bc5c74de service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:41:14.265801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:41:14.267078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:41:14.267406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:41:14.267685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:41:14.269471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:41:14.314137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:41:14.315661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:41:14.327370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1108/2227] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:41:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:41:14.343131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:41:14.344632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:41:14.353730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1120/2228] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:41:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:41:15.501446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:41:15.509982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cbaac-502e-40d5-93a3-c8dfe6d9b250 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:41:15.513621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:41:15.514260 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:41:15.514544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:41:15.514822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:41:15.517169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:41:15.530350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:41:15.532404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:41:15.543167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1109/2229] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:41:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:41:16.278765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:41:16.288023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30eca2-97ce-442d-86c5-50711cef2929 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:41:16.291558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:41:16.292204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:41:16.292489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:41:16.292788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:41:16.294541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:41:16.365031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:41:16.366412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:41:16.376835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1121/2230] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:41:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:41:25.538497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:41:26.372104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:42:00.979894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:42:00.981307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:42:00.995464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742a759-bee7-423c-a0cb-7b721242e704 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:42:01.000862 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:42:01.001496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:42:01.001819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:42:01.002096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:42:01.003986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:42:01.081312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:42:01.082915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:42:01.101071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1110/2231] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:42:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:42:11.098482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=41 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:42:18.160333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:42:18.161768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:42:18.178039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713c7a-cec0-4827-ba93-6f42812b0af1 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:42:18.183675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:42:18.184336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:42:18.184623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:42:18.185076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:42:18.186942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:42:18.200865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:42:18.202492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:42:18.214307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1122/2232] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:42:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:42:28.209487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:42:34.238237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:42:34.247727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8e43a-fa3e-48a1-97b3-c73c6af5a72c service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:42:34.251307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fdc2e-cb9f-4b77-88a3-38eac2afc13f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:42:34.251949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fdc2e-cb9f-4b77-88a3-38eac2afc13f service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:42:34.252232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fdc2e-cb9f-4b77-88a3-38eac2afc13f service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:42:34.252510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fdc2e-cb9f-4b77-88a3-38eac2afc13f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:42:34.261099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fdc2e-cb9f-4b77-88a3-38eac2afc13f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:42:34.262511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fdc2e-cb9f-4b77-88a3-38eac2afc13f service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:42:34.272015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1111/2233] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:42:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:42:44.267304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:42:58.952628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:42:58.962235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b726f-8724-4cfe-8a2f-fe68e121f190 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:42:58.965715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:42:58.966376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:42:58.966658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:42:58.966941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:42:58.968693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:42:58.978454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:42:58.979896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:42:58.987680 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1123/2234] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:42:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:43:08.984224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:45:05.229159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fdc2e-cb9f-4b77-88a3-38eac2afc13f service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:45:05.288591 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fdc2e-cb9f-4b77-88a3-38eac2afc13f service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:45:05.293636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd8e2e-fc22-4ac1-8778-42fd5f7efec2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:45:05.294268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd8e2e-fc22-4ac1-8778-42fd5f7efec2 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:45:05.294547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd8e2e-fc22-4ac1-8778-42fd5f7efec2 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:45:05.294821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd8e2e-fc22-4ac1-8778-42fd5f7efec2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:45:05.303131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd8e2e-fc22-4ac1-8778-42fd5f7efec2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:45:05.304743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd8e2e-fc22-4ac1-8778-42fd5f7efec2 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:45:05.314285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1112/2235] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:45:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:45:15.310204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=27 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:46:14.016695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:46:14.064747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f36df-6ef8-4a4a-9cde-10296fbd1e1c service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:46:14.070412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047bc33-6567-4d09-8c97-6263d3f06459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:46:14.071061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047bc33-6567-4d09-8c97-6263d3f06459 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:46:14.071381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047bc33-6567-4d09-8c97-6263d3f06459 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:46:14.071662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047bc33-6567-4d09-8c97-6263d3f06459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:46:14.080059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047bc33-6567-4d09-8c97-6263d3f06459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:46:14.081632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047bc33-6567-4d09-8c97-6263d3f06459 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:46:14.108336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1124/2236] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:46:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:46:14.939065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd8e2e-fc22-4ac1-8778-42fd5f7efec2 service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:46:14.951599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd8e2e-fc22-4ac1-8778-42fd5f7efec2 service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:46:14.956915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304c0b7-6967-4708-9df3-3808842a4655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:46:14.957653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304c0b7-6967-4708-9df3-3808842a4655 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:46:14.957959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304c0b7-6967-4708-9df3-3808842a4655 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:46:14.958248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304c0b7-6967-4708-9df3-3808842a4655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:46:14.966174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304c0b7-6967-4708-9df3-3808842a4655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:46:14.967791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304c0b7-6967-4708-9df3-3808842a4655 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:46:14.995160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1113/2237] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:46:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:46:15.264295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047bc33-6567-4d09-8c97-6263d3f06459 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:46:15.283025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047bc33-6567-4d09-8c97-6263d3f06459 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:46:15.286621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f5f84-d3f4-4ce0-a416-8350c34c0fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:46:15.287280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5f84-d3f4-4ce0-a416-8350c34c0fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:46:15.287568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5f84-d3f4-4ce0-a416-8350c34c0fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:46:15.287843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5f84-d3f4-4ce0-a416-8350c34c0fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:46:15.297957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f5f84-d3f4-4ce0-a416-8350c34c0fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:46:15.299417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f5f84-d3f4-4ce0-a416-8350c34c0fd0 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:46:15.310708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1125/2238] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:46:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:46:16.220068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304c0b7-6967-4708-9df3-3808842a4655 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:46:16.230038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304c0b7-6967-4708-9df3-3808842a4655 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:46:16.233695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cfe6f-ea7e-4f60-b7a0-e453768bfc64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:46:16.234332 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cfe6f-ea7e-4f60-b7a0-e453768bfc64 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:46:16.234615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cfe6f-ea7e-4f60-b7a0-e453768bfc64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:46:16.234891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cfe6f-ea7e-4f60-b7a0-e453768bfc64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:46:16.245577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cfe6f-ea7e-4f60-b7a0-e453768bfc64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:46:16.247032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cfe6f-ea7e-4f60-b7a0-e453768bfc64 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:46:16.258753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1114/2239] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:46:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:46:25.305503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=38 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:46:26.253342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=19 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:47:16.334009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f5f84-d3f4-4ce0-a416-8350c34c0fd0 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:47:16.353491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f5f84-d3f4-4ce0-a416-8350c34c0fd0 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:47:16.357050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5806eb-5bd9-4214-bb31-56173c2b8080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:47:16.357688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5806eb-5bd9-4214-bb31-56173c2b8080 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:47:16.357975 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5806eb-5bd9-4214-bb31-56173c2b8080 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:47:16.358250 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5806eb-5bd9-4214-bb31-56173c2b8080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:47:16.366822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5806eb-5bd9-4214-bb31-56173c2b8080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:47:16.368264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5806eb-5bd9-4214-bb31-56173c2b8080 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:47:16.394585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1126/2240] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:47:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:47:16.436464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cfe6f-ea7e-4f60-b7a0-e453768bfc64 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:47:16.445378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cfe6f-ea7e-4f60-b7a0-e453768bfc64 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:47:16.449863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:47:16.450496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:47:16.450775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:47:16.451071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:47:16.459463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:47:16.460865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:47:16.490346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1115/2241] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:47:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:47:16.532437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5806eb-5bd9-4214-bb31-56173c2b8080 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:47:16.540824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5806eb-5bd9-4214-bb31-56173c2b8080 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:47:16.544412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a12aa-fa95-45c2-82e8-bcbfc9d271bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:47:16.545052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a12aa-fa95-45c2-82e8-bcbfc9d271bf service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:47:16.545332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a12aa-fa95-45c2-82e8-bcbfc9d271bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:47:16.545606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a12aa-fa95-45c2-82e8-bcbfc9d271bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:47:16.554209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a12aa-fa95-45c2-82e8-bcbfc9d271bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:47:16.555978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a12aa-fa95-45c2-82e8-bcbfc9d271bf service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:47:16.565156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1127/2242] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:47:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:47:26.487590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=13 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:47:26.560896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=23 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:48:02.959167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:48:02.960888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:48:03.039415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ee237-b63d-46b3-a2e1-032fbb48e34e service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:48:03.042970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de0682-087f-47b5-b61a-07d565047e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:48:03.043645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de0682-087f-47b5-b61a-07d565047e5a service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:48:03.043926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de0682-087f-47b5-b61a-07d565047e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:48:03.044198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de0682-087f-47b5-b61a-07d565047e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:48:03.052318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de0682-087f-47b5-b61a-07d565047e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:48:03.053749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de0682-087f-47b5-b61a-07d565047e5a service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:48:03.062758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1116/2243] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:48:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:48:13.059093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=35 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:49:17.410874 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a12aa-fa95-45c2-82e8-bcbfc9d271bf service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:17.422594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a12aa-fa95-45c2-82e8-bcbfc9d271bf service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:17.422886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:17.423563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:17.423854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:17.424138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:17.425997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:17.506311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:17.507789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:17.527564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1128/2244] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:49:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:49:17.818724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf3700-42de-4e30-9ee1-7f643f430e68 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:17.820970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf3700-42de-4e30-9ee1-7f643f430e68 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:17.820970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf3700-42de-4e30-9ee1-7f643f430e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:17.845818 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-eebf3700-42de-4e30-9ee1-7f643f430e68 None None] MFA Rules not processed for user `15ff2cacb5c1450482597a2f5fa3c8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:49:17.847073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-eebf3700-42de-4e30-9ee1-7f643f430e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:17.854219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1117/2245] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:49:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:49:18.152118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:18.153672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:18.165068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1086ba5-4696-477e-861a-f9bfbd1dbfbc service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:18.169291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:18.169928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:18.170210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:18.170485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:18.172308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:18.181330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:18.182743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:18.190602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1129/2246] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:49:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:49:18.555840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf3700-42de-4e30-9ee1-7f643f430e68 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:18.567815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf3700-42de-4e30-9ee1-7f643f430e68 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:18.572991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6c826-6150-4117-9606-ebf2f66afb44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:18.574241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c826-6150-4117-9606-ebf2f66afb44 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:18.574528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c826-6150-4117-9606-ebf2f66afb44 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:18.574804 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c826-6150-4117-9606-ebf2f66afb44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:18.576588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e5d6c826-6150-4117-9606-ebf2f66afb44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:18.604769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6c826-6150-4117-9606-ebf2f66afb44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:18.604769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6c826-6150-4117-9606-ebf2f66afb44 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:18.613276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1118/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:49:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:49:22.048081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:22.063426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3c790-72fb-4f41-94fa-8272a4ddd7ff service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:22.072952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57764e16-aa3c-4c16-b8b4-2650fc155976 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:22.073812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57764e16-aa3c-4c16-b8b4-2650fc155976 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:22.074159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57764e16-aa3c-4c16-b8b4-2650fc155976 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:22.074506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57764e16-aa3c-4c16-b8b4-2650fc155976 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:22.085334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57764e16-aa3c-4c16-b8b4-2650fc155976 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:22.087268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57764e16-aa3c-4c16-b8b4-2650fc155976 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:22.099697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1130/2248] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:49:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:49:26.803658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54250e5-6119-4d2c-9d8c-f4a31f017f8e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:26.804163 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54250e5-6119-4d2c-9d8c-f4a31f017f8e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:26.804287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54250e5-6119-4d2c-9d8c-f4a31f017f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:26.831246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b54250e5-6119-4d2c-9d8c-f4a31f017f8e None None] MFA Rules not processed for user `cbcc15c786fb43669ac34a4897735c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:49:26.832480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b54250e5-6119-4d2c-9d8c-f4a31f017f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:26.840065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1119/2249] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:49:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:49:27.703851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57764e16-aa3c-4c16-b8b4-2650fc155976 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:27.716492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57764e16-aa3c-4c16-b8b4-2650fc155976 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:27.722625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:27.723542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:27.723809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:27.724180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:27.726164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:27.753711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:27.755231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:27.766252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1131/2250] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:49:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:49:30.675394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54250e5-6119-4d2c-9d8c-f4a31f017f8e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:30.685516 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54250e5-6119-4d2c-9d8c-f4a31f017f8e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:30.689192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:30.689839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:30.690119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:30.690390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:30.692609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:30.783075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:30.794279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:30.796235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:30.800998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ecf30-16a2-4316-ade7-d55256bd650d service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:30.804481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:30.805684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:30.805979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:30.806262 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:30.810300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:30.819112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1120/2251] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:49:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:49:30.824547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:30.826128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:30.839200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1132/2252] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:49:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:49:34.753212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:34.769618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95cb5-5337-4d7f-8fdc-d3323aa4dcdb service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:34.776854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:34.777648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:34.777998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:34.778347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:34.780259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:34.794738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:34.796517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:34.813702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1121/2253] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:49:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:49:34.893318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:34.894776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:34.908770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4618e-9b13-4b04-a4b6-5917614c3dc7 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:34.913302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:34.915921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:34.915921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:34.915921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] PATH_INFO: `/v3/users/e47c365946cf4264a2d2205ace04379d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:34.915921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e47c365946cf4264a2d2205ace04379d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:34.918327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:35.078356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e47c365946cf4264a2d2205ace04379d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.079052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e47c365946cf4264a2d2205ace04379d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.079752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e47c365946cf4264a2d2205ace04379d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.086825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e47c365946cf4264a2d2205ace04379d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.091192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:49:35.098754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-699ab541-ee66-4eaa-ae8b-3e7e75bdfc1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e47c365946cf4264a2d2205ace04379d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.178903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1133/2254] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:49:34 2022] DELETE /identity/v3/users/e47c365946cf4264a2d2205ace04379d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:49:35.425974 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:35.439418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22631a7-bbcf-400f-86a4-4177a757ec21 service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:35.495043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:35.495946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:35.496247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:35.496515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] PATH_INFO: `/v3/projects/c8639f8c836d4ad79d147b855d9017f8` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:35.497716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8639f8c836d4ad79d147b855d9017f8)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:35.499683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:35.502585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:49:35.614820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8639f8c836d4ad79d147b855d9017f8', 'request_id': 'req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.620346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8639f8c836d4ad79d147b855d9017f8', 'request_id': 'req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.716393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8639f8c836d4ad79d147b855d9017f8', 'request_id': 'req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.727891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] The token cache is being invalidate because project c8639f8c836d4ad79d147b855d9017f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:49:35.728177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.728830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c8f19fe-0d69-49f9-b3ec-c70f5d2343a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:49:35.730389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1122/2255] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:49:35 2022] DELETE /identity/v3/projects/c8639f8c836d4ad79d147b855d9017f8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:49:35.736239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31e1f-4af4-4fda-b94a-468e248851e9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:35.736532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31e1f-4af4-4fda-b94a-468e248851e9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:35.736810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd31e1f-4af4-4fda-b94a-468e248851e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:35.795175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bdd31e1f-4af4-4fda-b94a-468e248851e9 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:49:35.796520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bdd31e1f-4af4-4fda-b94a-468e248851e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:35.855930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1134/2256] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:49:35.863748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b60b1a-025c-45c1-827f-4ca553028511 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:35.864240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b60b1a-025c-45c1-827f-4ca553028511 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:35.864355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b60b1a-025c-45c1-827f-4ca553028511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:35.883957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-11b60b1a-025c-45c1-827f-4ca553028511 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:49:35.885712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-11b60b1a-025c-45c1-827f-4ca553028511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:35.893622 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1123/2257] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:49:35.897785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd31e1f-4af4-4fda-b94a-468e248851e9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:35.908534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd31e1f-4af4-4fda-b94a-468e248851e9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:35.913023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:35.913673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:35.913956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:35.914231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:35.914980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:49:35.915309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:35.916916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:35.928175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1135/2258] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:49:35.932407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b60b1a-025c-45c1-827f-4ca553028511 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:35.941503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b60b1a-025c-45c1-827f-4ca553028511 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:35.946035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a031c4a-e352-40f9-a47a-b81e378ab6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:35.946677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031c4a-e352-40f9-a47a-b81e378ab6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:35.946957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031c4a-e352-40f9-a47a-b81e378ab6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:35.947232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031c4a-e352-40f9-a47a-b81e378ab6a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:35.948471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a031c4a-e352-40f9-a47a-b81e378ab6a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:35.950330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a031c4a-e352-40f9-a47a-b81e378ab6a6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.038538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1124/2259] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:49:35 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:49:36.042644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:36.057528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f42e4-a5f5-4ea8-b1ec-66eb84bf5910 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:36.127704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce3cbc-174f-4cbb-ac8b-acccbf8f15d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:36.128397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce3cbc-174f-4cbb-ac8b-acccbf8f15d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.128687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce3cbc-174f-4cbb-ac8b-acccbf8f15d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.128970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce3cbc-174f-4cbb-ac8b-acccbf8f15d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.129683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce3cbc-174f-4cbb-ac8b-acccbf8f15d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:36.131704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce3cbc-174f-4cbb-ac8b-acccbf8f15d2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.196391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1136/2260] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:49:36 2022] POST /identity/v3/users => generated 303 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:49:36.201581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a031c4a-e352-40f9-a47a-b81e378ab6a6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:36.211756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a031c4a-e352-40f9-a47a-b81e378ab6a6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:36.216405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97134992-389f-40d6-bd4c-b52d6c2c597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:36.217056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97134992-389f-40d6-bd4c-b52d6c2c597a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.217340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97134992-389f-40d6-bd4c-b52d6c2c597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.217635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97134992-389f-40d6-bd4c-b52d6c2c597a admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.218334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97134992-389f-40d6-bd4c-b52d6c2c597a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:36.219952 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97134992-389f-40d6-bd4c-b52d6c2c597a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.229775 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1125/2261] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:49:36.233463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce3cbc-174f-4cbb-ac8b-acccbf8f15d2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:36.243429 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce3cbc-174f-4cbb-ac8b-acccbf8f15d2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:36.248487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790dc296-623e-4b01-b775-2ca33251a629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:36.249138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dc296-623e-4b01-b775-2ca33251a629 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.249421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dc296-623e-4b01-b775-2ca33251a629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.249695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790dc296-623e-4b01-b775-2ca33251a629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.250407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790dc296-623e-4b01-b775-2ca33251a629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:36.252027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790dc296-623e-4b01-b775-2ca33251a629 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.262174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1137/2262] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:49:36.266025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97134992-389f-40d6-bd4c-b52d6c2c597a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:36.275767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97134992-389f-40d6-bd4c-b52d6c2c597a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:36.286744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396efba-a598-478b-983b-bd96aaa9a3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:36.287720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396efba-a598-478b-983b-bd96aaa9a3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.287998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396efba-a598-478b-983b-bd96aaa9a3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.288271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396efba-a598-478b-983b-bd96aaa9a3aa admin admin] PATH_INFO: `/v3/projects/aed12fb2e6dd46589003991db00105df/users/d370ca777eec461e86390ee7834e79d1/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.303677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396efba-a598-478b-983b-bd96aaa9a3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed12fb2e6dd46589003991db00105df, user_id=d370ca777eec461e86390ee7834e79d1, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:36.306349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396efba-a598-478b-983b-bd96aaa9a3aa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.384844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1126/2263] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:49:36 2022] PUT /identity/v3/projects/aed12fb2e6dd46589003991db00105df/users/d370ca777eec461e86390ee7834e79d1/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:49:36.396140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04105236-e2af-4c42-97a3-eac7f2d22c45 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.396438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04105236-e2af-4c42-97a3-eac7f2d22c45 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.396740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04105236-e2af-4c42-97a3-eac7f2d22c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.420843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:49:36.424129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-04105236-e2af-4c42-97a3-eac7f2d22c45 None None] MFA Rules not processed for user `d370ca777eec461e86390ee7834e79d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:49:36.425346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-04105236-e2af-4c42-97a3-eac7f2d22c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:36.488330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1138/2264] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:49:36 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:49:36.492891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396efba-a598-478b-983b-bd96aaa9a3aa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:36.554266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396efba-a598-478b-983b-bd96aaa9a3aa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:36.559528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f2ae-a668-4162-b0f3-3e768c9dff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:36.560179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f2ae-a668-4162-b0f3-3e768c9dff12 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.560464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f2ae-a668-4162-b0f3-3e768c9dff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.560742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09f2ae-a668-4162-b0f3-3e768c9dff12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.561427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09f2ae-a668-4162-b0f3-3e768c9dff12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:36.563337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09f2ae-a668-4162-b0f3-3e768c9dff12 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.672074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1127/2265] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:49:36 2022] POST /identity/v3/projects => generated 405 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:49:36.676197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04105236-e2af-4c42-97a3-eac7f2d22c45 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:36.687426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04105236-e2af-4c42-97a3-eac7f2d22c45 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:36.741687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567db309-8110-4aa4-9e16-3abe82c2a488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:36.742282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567db309-8110-4aa4-9e16-3abe82c2a488 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.742564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567db309-8110-4aa4-9e16-3abe82c2a488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.742839 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567db309-8110-4aa4-9e16-3abe82c2a488 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.743569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567db309-8110-4aa4-9e16-3abe82c2a488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:36.746019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567db309-8110-4aa4-9e16-3abe82c2a488 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.824488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1139/2266] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:49:36 2022] POST /identity/v3/users => generated 309 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:49:36.828384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f2ae-a668-4162-b0f3-3e768c9dff12 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:36.834909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:49:36.839279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09f2ae-a668-4162-b0f3-3e768c9dff12 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:36.843939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801d530-d78b-4fd1-99f1-b657b274bd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:36.844589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801d530-d78b-4fd1-99f1-b657b274bd5a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.844870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801d530-d78b-4fd1-99f1-b657b274bd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.845162 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801d530-d78b-4fd1-99f1-b657b274bd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.845888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801d530-d78b-4fd1-99f1-b657b274bd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:36.847540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801d530-d78b-4fd1-99f1-b657b274bd5a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.858490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1128/2267] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:49:36.862493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567db309-8110-4aa4-9e16-3abe82c2a488 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:36.872578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567db309-8110-4aa4-9e16-3abe82c2a488 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:36.877160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2ec88-c2cb-48f8-992a-f610345458da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:36.878037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2ec88-c2cb-48f8-992a-f610345458da admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:36.878323 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2ec88-c2cb-48f8-992a-f610345458da admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:36.878602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2ec88-c2cb-48f8-992a-f610345458da admin admin] PATH_INFO: `/v3/projects/00353860871c4e16b4d9c8798fa5fa09/users/7cdfbbb5387c43e2b15aba01e4f649bf/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:36.894091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2ec88-c2cb-48f8-992a-f610345458da admin admin] RBAC: Authorizing `identity:create_grant(project_id=00353860871c4e16b4d9c8798fa5fa09, user_id=7cdfbbb5387c43e2b15aba01e4f649bf, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:36.896796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2ec88-c2cb-48f8-992a-f610345458da admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:36.985756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1140/2268] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:49:36 2022] PUT /identity/v3/projects/00353860871c4e16b4d9c8798fa5fa09/users/7cdfbbb5387c43e2b15aba01e4f649bf/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:49:36.989696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801d530-d78b-4fd1-99f1-b657b274bd5a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:37.000199 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801d530-d78b-4fd1-99f1-b657b274bd5a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:37.056310 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23c87a-eda7-42c9-b952-29da1569e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:37.057007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c87a-eda7-42c9-b952-29da1569e946 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:37.057291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c87a-eda7-42c9-b952-29da1569e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:37.057567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23c87a-eda7-42c9-b952-29da1569e946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:37.058297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23c87a-eda7-42c9-b952-29da1569e946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:37.059941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23c87a-eda7-42c9-b952-29da1569e946 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:37.069877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1129/2269] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:49:37.074775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2ec88-c2cb-48f8-992a-f610345458da admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:49:37.084609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2ec88-c2cb-48f8-992a-f610345458da admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:49:37.089662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:49:37.090540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:37.090823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:37.091102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] PATH_INFO: `/v3/projects/00353860871c4e16b4d9c8798fa5fa09/users/7cdfbbb5387c43e2b15aba01e4f649bf/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:37.093484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00353860871c4e16b4d9c8798fa5fa09, user_id=7cdfbbb5387c43e2b15aba01e4f649bf, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:49:37.096202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:49:37.106912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1141/2270] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:49:37 2022] PUT /identity/v3/projects/00353860871c4e16b4d9c8798fa5fa09/users/7cdfbbb5387c43e2b15aba01e4f649bf/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:49:37.117464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374983b-6eb9-4031-baef-31d0565145c5 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:49:37.117750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374983b-6eb9-4031-baef-31d0565145c5 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:49:37.118047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374983b-6eb9-4031-baef-31d0565145c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:49:37.147781 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1374983b-6eb9-4031-baef-31d0565145c5 None None] MFA Rules not processed for user `7cdfbbb5387c43e2b15aba01e4f649bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:49:37.149624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1374983b-6eb9-4031-baef-31d0565145c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:49:37.217953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1130/2271] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:49:37 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:49:47.102497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:49:47.214962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=66 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:50:07.200900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:50:07.202677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:50:07.270028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc195e1f-b8ac-4195-a307-7541f9696a20 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:50:07.274869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:50:07.275524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:50:07.275807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:50:07.276082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:50:07.277850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:50:07.288710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:50:07.290264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:50:07.300346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1142/2272] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:50:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:50:17.295760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=32 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:51:17.909595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374983b-6eb9-4031-baef-31d0565145c5 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:17.911602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1374983b-6eb9-4031-baef-31d0565145c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:17.977839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374983b-6eb9-4031-baef-31d0565145c5 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:17.981617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:17.982273 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:17.982558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:17.982835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:17.992569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:17.994002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:18.003402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1131/2273] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:51:17 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:51:18.037041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:18.046748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30860f-de4f-4745-a77c-cb8ea224fb34 service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:18.050455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:18.051095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:18.051412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:18.051721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:18.053507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:18.130751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:18.132215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:18.141192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1143/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:51:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:51:18.302813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:18.304220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:18.377360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1dd83-90b7-4932-941c-3c27d6e5dae3 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:18.383181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:18.383894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:18.384181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:18.384464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:18.387602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:18.460478 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:18.462168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:18.471141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1132/2275] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:51:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:51:18.891659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:18.893037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:18.954521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494434d7-4bc2-4967-ad46-432a94acdff8 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:18.958290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b81db6-1017-4463-bfa6-5e20e53e4f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:18.958944 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b81db6-1017-4463-bfa6-5e20e53e4f41 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:18.959233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b81db6-1017-4463-bfa6-5e20e53e4f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:18.959545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b81db6-1017-4463-bfa6-5e20e53e4f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:18.961356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-21b81db6-1017-4463-bfa6-5e20e53e4f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:18.974597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b81db6-1017-4463-bfa6-5e20e53e4f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:18.976105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b81db6-1017-4463-bfa6-5e20e53e4f41 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:18.987166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1144/2276] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:51:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:51:20.173210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:20.174718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:20.186305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b386a20-7e58-4522-b0bf-735aaa1cd5ae service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:20.190155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:20.190786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:20.191068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:20.191381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:20.193710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:20.207482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:20.209620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:20.220579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1133/2277] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:51:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:51:25.702240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26340e-2fc3-49a9-babc-b3c1e714ae53 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:25.702721 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26340e-2fc3-49a9-babc-b3c1e714ae53 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:25.702815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26340e-2fc3-49a9-babc-b3c1e714ae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:25.767499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8e26340e-2fc3-49a9-babc-b3c1e714ae53 None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:51:25.768654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8e26340e-2fc3-49a9-babc-b3c1e714ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:25.826170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:51:25.836921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1145/2278] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 11 11:51:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:51:25.879810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:25.881792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:25.886200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:51:25.973875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6331517-fb37-4bf8-af37-823f5eaa2d48 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:25.979834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbac7b3-ac34-4328-b8e5-1d718bb62c30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:25.980487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac7b3-ac34-4328-b8e5-1d718bb62c30 service swift] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:25.980771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac7b3-ac34-4328-b8e5-1d718bb62c30 service swift] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:25.981050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbac7b3-ac34-4328-b8e5-1d718bb62c30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:25.989490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbac7b3-ac34-4328-b8e5-1d718bb62c30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:25.991555 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbac7b3-ac34-4328-b8e5-1d718bb62c30 service swift] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:25.999661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1134/2279] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 11:51:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 11:51:26.092872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f41aee-5400-48ab-9e1d-e90c8609e9ba None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:26.094732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f41aee-5400-48ab-9e1d-e90c8609e9ba None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:26.094732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f41aee-5400-48ab-9e1d-e90c8609e9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:26.110517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-18f41aee-5400-48ab-9e1d-e90c8609e9ba None None] MFA Rules not processed for user `cbcc15c786fb43669ac34a4897735c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:51:26.111751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-18f41aee-5400-48ab-9e1d-e90c8609e9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:26.118593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1146/2280] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:51:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:51:26.425479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbac7b3-ac34-4328-b8e5-1d718bb62c30 service swift] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:26.434590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbac7b3-ac34-4328-b8e5-1d718bb62c30 service swift] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:26.438203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:26.438885 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:26.439119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:26.439423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:26.444180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:26.453500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:26.455066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:26.463763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1135/2281] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:51:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:51:27.019007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f41aee-5400-48ab-9e1d-e90c8609e9ba None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:27.029199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f41aee-5400-48ab-9e1d-e90c8609e9ba None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:27.032704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:27.033337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:27.033618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:27.033897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:27.035638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:27.104732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:27.106163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:27.116418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1147/2282] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:51:30.884177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:30.885716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:30.900824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a780b73-4eac-4096-bb8c-d655f84371ee service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:30.905964 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:30.906605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:30.906890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:30.907171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:30.908958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:30.921182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:30.922843 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:30.932736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1136/2283] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:51:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:51:34.805423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:34.806924 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:34.820612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7e0ce-982e-458d-af34-8a51ba2f5789 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:34.825599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:34.826457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:34.826741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:34.827017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] PATH_INFO: `/v3/users/559a564c171f4dd1abb3e4af27987bbd` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:34.828257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=559a564c171f4dd1abb3e4af27987bbd)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:34.830167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:35.012712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '559a564c171f4dd1abb3e4af27987bbd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.013301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '559a564c171f4dd1abb3e4af27987bbd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.013910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '559a564c171f4dd1abb3e4af27987bbd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.021414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '559a564c171f4dd1abb3e4af27987bbd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.033245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '559a564c171f4dd1abb3e4af27987bbd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.113407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1148/2284] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:51:34 2022] DELETE /identity/v3/users/559a564c171f4dd1abb3e4af27987bbd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:51:35.639581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:35.650902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f9a78-0891-49da-9e86-edf834d5bdc1 service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:35.720016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:35.720877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:35.721159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:35.721431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] PATH_INFO: `/v3/projects/72a461d4496f4015969f2683af43afa7` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:35.722604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a461d4496f4015969f2683af43afa7)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:35.724564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:35.855773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a461d4496f4015969f2683af43afa7', 'request_id': 'req-e2fe270e-e987-4a4e-a5e9-e413dc01db68'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.856924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:51:35.861720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a461d4496f4015969f2683af43afa7', 'request_id': 'req-e2fe270e-e987-4a4e-a5e9-e413dc01db68'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.919239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a461d4496f4015969f2683af43afa7', 'request_id': 'req-e2fe270e-e987-4a4e-a5e9-e413dc01db68'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.931443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] The token cache is being invalidate because project 72a461d4496f4015969f2683af43afa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:51:35.931741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.932371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:35.934036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1137/2285] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:51:35 2022] DELETE /identity/v3/projects/72a461d4496f4015969f2683af43afa7 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:51:35.938124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:35.939577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:35.952699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a25234-ed7c-43d7-b89f-d345c56a2b05 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:36.007755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:51:36.016358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:36.017231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:36.017514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:36.017789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] PATH_INFO: `/v3/users/cea34f32015d4694896f3266e2218d93` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:36.031377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea34f32015d4694896f3266e2218d93)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:36.034197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:36.259159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea34f32015d4694896f3266e2218d93'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.259849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea34f32015d4694896f3266e2218d93'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.260468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea34f32015d4694896f3266e2218d93'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.267286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea34f32015d4694896f3266e2218d93'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.279009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea34f32015d4694896f3266e2218d93'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.366829 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1149/2286] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:51:35 2022] DELETE /identity/v3/users/cea34f32015d4694896f3266e2218d93 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:51:36.610606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:51:36.612117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:51:36.632168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe270e-e987-4a4e-a5e9-e413dc01db68 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:51:36.701896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:51:36.702755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:51:36.703151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:51:36.703369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] PATH_INFO: `/v3/projects/0b4f15a303e245d4ab443dcd83e82bfd` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:51:36.713792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4f15a303e245d4ab443dcd83e82bfd)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:51:36.715762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:51:36.809005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4f15a303e245d4ab443dcd83e82bfd', 'request_id': 'req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.814253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4f15a303e245d4ab443dcd83e82bfd', 'request_id': 'req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.896384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4f15a303e245d4ab443dcd83e82bfd', 'request_id': 'req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.908252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] The token cache is being invalidate because project 0b4f15a303e245d4ab443dcd83e82bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:51:36.908541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.909162 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:51:36.910702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1138/2287] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:51:36 2022] DELETE /identity/v3/projects/0b4f15a303e245d4ab443dcd83e82bfd => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:51:46.662766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:51:46.905271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=10,keystone:SELECT=61,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:06.973841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:06.975472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:07.031084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72738c90-15ed-4fa3-9b1b-dd2919f1e652 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:07.079794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:07.080453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:07.080740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:07.081022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:07.082822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:07.091987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:07.093406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:07.101713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1150/2288] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:53:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:53:17.097723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=46 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:19.563097 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:19.564844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:19.622214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c90e1-d903-47dc-8d0a-4dc35951e5d0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:19.625794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:19.627826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.627826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.627826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.628805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:19.699195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:19.700671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:19.709698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1139/2289] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:53:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:53:19.752459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:19.762211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9f2c0-2935-48ee-8c97-5a16f5450d73 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:19.769768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284242f-d1eb-48e0-877a-8a5d4c866b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:19.769768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284242f-d1eb-48e0-877a-8a5d4c866b89 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:19.769768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284242f-d1eb-48e0-877a-8a5d4c866b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:19.769768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284242f-d1eb-48e0-877a-8a5d4c866b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:19.770211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e284242f-d1eb-48e0-877a-8a5d4c866b89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:19.848710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284242f-d1eb-48e0-877a-8a5d4c866b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:19.850145 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284242f-d1eb-48e0-877a-8a5d4c866b89 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:19.868536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1151/2290] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:53:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:53:20.604897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:20.606307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:20.673855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aeea39-64e9-475e-bbbd-1ceff37c0731 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:20.678528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:20.679457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.679745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.680020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] PATH_INFO: `/v3/users/d370ca777eec461e86390ee7834e79d1` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.681227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d370ca777eec461e86390ee7834e79d1)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:20.683133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:20.865466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:20.867043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd370ca777eec461e86390ee7834e79d1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:20.867721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd370ca777eec461e86390ee7834e79d1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:20.868312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd370ca777eec461e86390ee7834e79d1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:20.875174 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd370ca777eec461e86390ee7834e79d1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:20.887388 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd370ca777eec461e86390ee7834e79d1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:20.944967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1140/2291] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:53:20 2022] DELETE /identity/v3/users/d370ca777eec461e86390ee7834e79d1 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:20.949903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140493d5-ba74-4e9e-8836-b383adae6090 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:20.950392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140493d5-ba74-4e9e-8836-b383adae6090 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:20.950520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140493d5-ba74-4e9e-8836-b383adae6090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:20.967001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-140493d5-ba74-4e9e-8836-b383adae6090 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:20.971116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-140493d5-ba74-4e9e-8836-b383adae6090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:21.026440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1152/2292] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:53:21.036003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.037379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:21.095105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabf3cb-3612-461f-89ac-d9cd0adc889b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.098757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2ebca-d11f-4ca8-86b0-db930c0611e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.099434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ebca-d11f-4ca8-86b0-db930c0611e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.099725 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ebca-d11f-4ca8-86b0-db930c0611e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.100003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2ebca-d11f-4ca8-86b0-db930c0611e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.108911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2ebca-d11f-4ca8-86b0-db930c0611e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.110340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e2ebca-d11f-4ca8-86b0-db930c0611e7 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.120296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1141/2293] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:53:21.539549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140493d5-ba74-4e9e-8836-b383adae6090 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.542688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-140493d5-ba74-4e9e-8836-b383adae6090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:21.556359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140493d5-ba74-4e9e-8836-b383adae6090 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.562591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.563528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.563820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.564554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] PATH_INFO: `/v3/projects/aed12fb2e6dd46589003991db00105df` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.565517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed12fb2e6dd46589003991db00105df)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.567781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.684420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed12fb2e6dd46589003991db00105df', 'request_id': 'req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:21.689970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed12fb2e6dd46589003991db00105df', 'request_id': 'req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:21.754051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed12fb2e6dd46589003991db00105df', 'request_id': 'req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:21.755347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:21.770451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] The token cache is being invalidate because project aed12fb2e6dd46589003991db00105df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:53:21.770738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:21.771418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:21.772965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1153/2294] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:53:21 2022] DELETE /identity/v3/projects/aed12fb2e6dd46589003991db00105df => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:21.776926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2ebca-d11f-4ca8-86b0-db930c0611e7 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:21.787460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2ebca-d11f-4ca8-86b0-db930c0611e7 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:21.843668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:21.844538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:21.844819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:21.845096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] PATH_INFO: `/v3/users/7cdfbbb5387c43e2b15aba01e4f649bf` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:21.846274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cdfbbb5387c43e2b15aba01e4f649bf)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:21.848890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:21.995415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cdfbbb5387c43e2b15aba01e4f649bf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:21.996073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cdfbbb5387c43e2b15aba01e4f649bf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:21.996672 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cdfbbb5387c43e2b15aba01e4f649bf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:22.003450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cdfbbb5387c43e2b15aba01e4f649bf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:22.011062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:22.015327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-47da463c-2dbf-4399-99bd-927d6e9d012a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cdfbbb5387c43e2b15aba01e4f649bf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:22.126136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1142/2295] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:53:21 2022] DELETE /identity/v3/users/7cdfbbb5387c43e2b15aba01e4f649bf => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:53:22.535034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:22.536560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:22.553922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f128c7-72a0-4517-a0a7-3ca5bd799cba admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:22.624333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:22.625245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:22.625538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:22.625977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] PATH_INFO: `/v3/projects/00353860871c4e16b4d9c8798fa5fa09` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.627018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=00353860871c4e16b4d9c8798fa5fa09)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:22.628989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:22.744881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00353860871c4e16b4d9c8798fa5fa09', 'request_id': 'req-933b8be5-42f1-44f2-a3d0-5a0c428483ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:22.750657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00353860871c4e16b4d9c8798fa5fa09', 'request_id': 'req-933b8be5-42f1-44f2-a3d0-5a0c428483ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:22.810701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00353860871c4e16b4d9c8798fa5fa09', 'request_id': 'req-933b8be5-42f1-44f2-a3d0-5a0c428483ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:22.822683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] The token cache is being invalidate because project 00353860871c4e16b4d9c8798fa5fa09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:53:22.822968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:22.823641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-933b8be5-42f1-44f2-a3d0-5a0c428483ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:53:22.825305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1154/2296] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:53:22 2022] DELETE /identity/v3/projects/00353860871c4e16b4d9c8798fa5fa09 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:22.830344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73cc38-597c-426b-ac5b-007b34bc1575 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:22.830850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73cc38-597c-426b-ac5b-007b34bc1575 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:22.830965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73cc38-597c-426b-ac5b-007b34bc1575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.849519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ab73cc38-597c-426b-ac5b-007b34bc1575 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:22.850869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ab73cc38-597c-426b-ac5b-007b34bc1575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:22.910678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1143/2297] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:53:22.926296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b461c-07c8-4a29-ba0d-ae226e18dc19 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:22.928324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b461c-07c8-4a29-ba0d-ae226e18dc19 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:22.929690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b461c-07c8-4a29-ba0d-ae226e18dc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.956276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f72b461c-07c8-4a29-ba0d-ae226e18dc19 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:22.959539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f72b461c-07c8-4a29-ba0d-ae226e18dc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:22.966215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1155/2298] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:53:22.970382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73cc38-597c-426b-ac5b-007b34bc1575 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:22.980472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73cc38-597c-426b-ac5b-007b34bc1575 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:22.985045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:22.988171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:22.988171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:22.988171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:22.988171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:53:22.988171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:22.989184 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.001104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1144/2299] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:53:23.005968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b461c-07c8-4a29-ba0d-ae226e18dc19 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.015375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b461c-07c8-4a29-ba0d-ae226e18dc19 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.020833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6a78-4c19-45ab-b241-86848168ad2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.021493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6a78-4c19-45ab-b241-86848168ad2b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.021777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6a78-4c19-45ab-b241-86848168ad2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.022055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf6a78-4c19-45ab-b241-86848168ad2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.022824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf6a78-4c19-45ab-b241-86848168ad2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.024905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf6a78-4c19-45ab-b241-86848168ad2b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.125304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1156/2300] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:53:23 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:23.128985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.140384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843220ed-e313-497c-a4d1-d9537ea28df0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.197693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaf92b-67ff-4ea9-9484-ddddc1649688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.198248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf92b-67ff-4ea9-9484-ddddc1649688 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.198537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf92b-67ff-4ea9-9484-ddddc1649688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.198813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf92b-67ff-4ea9-9484-ddddc1649688 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.199536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaf92b-67ff-4ea9-9484-ddddc1649688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.201450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaf92b-67ff-4ea9-9484-ddddc1649688 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.286348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1145/2301] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:53:23 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:23.290428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6a78-4c19-45ab-b241-86848168ad2b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.300736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf6a78-4c19-45ab-b241-86848168ad2b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.305447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b10f4-8f33-439c-aa70-9bf758adbe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.306116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b10f4-8f33-439c-aa70-9bf758adbe37 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.306411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b10f4-8f33-439c-aa70-9bf758adbe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.306692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b10f4-8f33-439c-aa70-9bf758adbe37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.307522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b10f4-8f33-439c-aa70-9bf758adbe37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.309169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b10f4-8f33-439c-aa70-9bf758adbe37 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.319761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1157/2302] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:53:23.323996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaf92b-67ff-4ea9-9484-ddddc1649688 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.335489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaf92b-67ff-4ea9-9484-ddddc1649688 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.340422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120e547-19ce-4456-8d06-6081419c27a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.341161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120e547-19ce-4456-8d06-6081419c27a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.341468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120e547-19ce-4456-8d06-6081419c27a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.341746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120e547-19ce-4456-8d06-6081419c27a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.342522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120e547-19ce-4456-8d06-6081419c27a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.344198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120e547-19ce-4456-8d06-6081419c27a9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.358204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1146/2303] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:53:23.359486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b10f4-8f33-439c-aa70-9bf758adbe37 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.372132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b10f4-8f33-439c-aa70-9bf758adbe37 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.377059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1143c5-a185-418e-9403-287dfaee132c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.378200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1143c5-a185-418e-9403-287dfaee132c admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.378484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1143c5-a185-418e-9403-287dfaee132c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.378762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1143c5-a185-418e-9403-287dfaee132c admin admin] PATH_INFO: `/v3/projects/14557e24f9f74819a96e5f61d0c4616b/users/6bb80a2610c64768a5ba9b008b8e9eb4/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.394019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1143c5-a185-418e-9403-287dfaee132c admin admin] RBAC: Authorizing `identity:create_grant(project_id=14557e24f9f74819a96e5f61d0c4616b, user_id=6bb80a2610c64768a5ba9b008b8e9eb4, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.397210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1143c5-a185-418e-9403-287dfaee132c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.469343 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1158/2304] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:53:23 2022] PUT /identity/v3/projects/14557e24f9f74819a96e5f61d0c4616b/users/6bb80a2610c64768a5ba9b008b8e9eb4/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:23.481390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3fa4f-422b-4c2a-8492-495bc2cac04e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.481683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3fa4f-422b-4c2a-8492-495bc2cac04e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.481966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3fa4f-422b-4c2a-8492-495bc2cac04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.509215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-81c3fa4f-422b-4c2a-8492-495bc2cac04e None None] MFA Rules not processed for user `6bb80a2610c64768a5ba9b008b8e9eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:23.510455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-81c3fa4f-422b-4c2a-8492-495bc2cac04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:23.551603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:23.576713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1147/2305] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:53:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:53:23.581283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1143c5-a185-418e-9403-287dfaee132c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.603354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:23.644614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1143c5-a185-418e-9403-287dfaee132c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.650528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfb0c6-5135-4dac-9ec5-51331540e7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.651227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfb0c6-5135-4dac-9ec5-51331540e7fb admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.651539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfb0c6-5135-4dac-9ec5-51331540e7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.651881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfb0c6-5135-4dac-9ec5-51331540e7fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.652579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfb0c6-5135-4dac-9ec5-51331540e7fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.654478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfb0c6-5135-4dac-9ec5-51331540e7fb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.762713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1159/2306] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:53:23 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:53:23.766939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3fa4f-422b-4c2a-8492-495bc2cac04e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.778659 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3fa4f-422b-4c2a-8492-495bc2cac04e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.838319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58d0a9-7ea3-4c73-a6cf-e459f2203549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.838996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58d0a9-7ea3-4c73-a6cf-e459f2203549 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.839187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58d0a9-7ea3-4c73-a6cf-e459f2203549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.839495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58d0a9-7ea3-4c73-a6cf-e459f2203549 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.840195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58d0a9-7ea3-4c73-a6cf-e459f2203549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.842114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58d0a9-7ea3-4c73-a6cf-e459f2203549 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.952565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1148/2307] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:53:23 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:53:23.954058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfb0c6-5135-4dac-9ec5-51331540e7fb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:23.964697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfb0c6-5135-4dac-9ec5-51331540e7fb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:23.969563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb7626-9e44-4337-b6c7-7a1f0c13f49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:23.970260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb7626-9e44-4337-b6c7-7a1f0c13f49f admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:23.970544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb7626-9e44-4337-b6c7-7a1f0c13f49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:23.970821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb7626-9e44-4337-b6c7-7a1f0c13f49f admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:23.971717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb7626-9e44-4337-b6c7-7a1f0c13f49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:23.973397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb7626-9e44-4337-b6c7-7a1f0c13f49f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:23.988266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1160/2308] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:53:23.992774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58d0a9-7ea3-4c73-a6cf-e459f2203549 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:24.003211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58d0a9-7ea3-4c73-a6cf-e459f2203549 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:24.009942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c85af-3f61-4989-8fd6-e8da6372dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:24.010837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c85af-3f61-4989-8fd6-e8da6372dee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:24.011119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c85af-3f61-4989-8fd6-e8da6372dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:24.011434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c85af-3f61-4989-8fd6-e8da6372dee3 admin admin] PATH_INFO: `/v3/projects/6051d0cdfc1f4a7bb251b54cb4aeff20/users/cb16394e90e2474eb30760b6e7e10798/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:24.033445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c85af-3f61-4989-8fd6-e8da6372dee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6051d0cdfc1f4a7bb251b54cb4aeff20, user_id=cb16394e90e2474eb30760b6e7e10798, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:24.038695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c85af-3f61-4989-8fd6-e8da6372dee3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:24.152083 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1149/2309] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:53:23 2022] PUT /identity/v3/projects/6051d0cdfc1f4a7bb251b54cb4aeff20/users/cb16394e90e2474eb30760b6e7e10798/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:24.156271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb7626-9e44-4337-b6c7-7a1f0c13f49f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:24.167603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb7626-9e44-4337-b6c7-7a1f0c13f49f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:24.241368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cd250-c320-435b-b7f6-f4be932cbdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:24.242244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cd250-c320-435b-b7f6-f4be932cbdbe admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:24.242752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cd250-c320-435b-b7f6-f4be932cbdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:24.242910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cd250-c320-435b-b7f6-f4be932cbdbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:24.243584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cd250-c320-435b-b7f6-f4be932cbdbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:24.245209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cd250-c320-435b-b7f6-f4be932cbdbe admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:24.256880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1161/2310] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:53:24.261146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c85af-3f61-4989-8fd6-e8da6372dee3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:24.273443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c85af-3f61-4989-8fd6-e8da6372dee3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:24.278295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:24.279212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:24.279528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:24.279807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] PATH_INFO: `/v3/projects/6051d0cdfc1f4a7bb251b54cb4aeff20/users/cb16394e90e2474eb30760b6e7e10798/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:24.281840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6051d0cdfc1f4a7bb251b54cb4aeff20, user_id=cb16394e90e2474eb30760b6e7e10798, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:24.284539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:24.296830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1150/2311] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:53:24 2022] PUT /identity/v3/projects/6051d0cdfc1f4a7bb251b54cb4aeff20/users/cb16394e90e2474eb30760b6e7e10798/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:53:24.307335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe37409-b687-440b-a4cb-ec5c84aff9b7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:24.307634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe37409-b687-440b-a4cb-ec5c84aff9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:24.311646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe37409-b687-440b-a4cb-ec5c84aff9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:24.337266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ebe37409-b687-440b-a4cb-ec5c84aff9b7 None None] MFA Rules not processed for user `cb16394e90e2474eb30760b6e7e10798`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:53:24.338497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ebe37409-b687-440b-a4cb-ec5c84aff9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:24.410848 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1162/2312] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:53:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:53:24.422049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:53:24.423782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:53:24.483495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b47284-5a40-4d48-8917-dc1cb6f8e0c9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:53:24.487506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:53:24.488190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:53:24.488471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:53:24.488745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:53:24.499272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:53:24.501548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:53:24.514484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1151/2313] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:53:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:53:34.407751 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:53:34.508803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:14.156047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe37409-b687-440b-a4cb-ec5c84aff9b7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:14.157418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ebe37409-b687-440b-a4cb-ec5c84aff9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:14.256337 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe37409-b687-440b-a4cb-ec5c84aff9b7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:14.261365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:14.262010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:14.262294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:14.262573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:14.264385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:14.275749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:14.277324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:14.287065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1163/2314] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:55:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:55:23.564514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:23.565133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:23.577144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f872ee-c8ba-4f4c-93e0-556897ef0c48 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:23.580879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc038a57-e39b-404d-b8c7-c1f9cbe5b58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:23.581514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038a57-e39b-404d-b8c7-c1f9cbe5b58e service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:23.581801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038a57-e39b-404d-b8c7-c1f9cbe5b58e service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:23.582077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038a57-e39b-404d-b8c7-c1f9cbe5b58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:23.637327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc038a57-e39b-404d-b8c7-c1f9cbe5b58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:23.638745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc038a57-e39b-404d-b8c7-c1f9cbe5b58e service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:23.648278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1152/2315] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:55:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:55:24.282841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=45 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:24.385062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:24.386455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:24.447188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c51b9-5278-4f51-bc29-08f8f595ee20 service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:24.450846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573888b9-af06-4d36-ab57-cf75a31d4fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:24.451564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573888b9-af06-4d36-ab57-cf75a31d4fde service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:24.451845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573888b9-af06-4d36-ab57-cf75a31d4fde service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:24.452116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573888b9-af06-4d36-ab57-cf75a31d4fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:24.453864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-573888b9-af06-4d36-ab57-cf75a31d4fde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:24.519621 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573888b9-af06-4d36-ab57-cf75a31d4fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:24.521316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573888b9-af06-4d36-ab57-cf75a31d4fde service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:24.531031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1164/2316] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:55:24.909300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc038a57-e39b-404d-b8c7-c1f9cbe5b58e service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:24.925705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc038a57-e39b-404d-b8c7-c1f9cbe5b58e service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:24.930353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:24.931417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:24.931874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:24.932324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] PATH_INFO: `/v3/users/6bb80a2610c64768a5ba9b008b8e9eb4` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:24.933613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb80a2610c64768a5ba9b008b8e9eb4)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:24.935741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:25.146161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb80a2610c64768a5ba9b008b8e9eb4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:25.147383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb80a2610c64768a5ba9b008b8e9eb4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:25.148057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb80a2610c64768a5ba9b008b8e9eb4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:25.155093 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb80a2610c64768a5ba9b008b8e9eb4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:25.167196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb80a2610c64768a5ba9b008b8e9eb4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:25.258435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1153/2317] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:55:24 2022] DELETE /identity/v3/users/6bb80a2610c64768a5ba9b008b8e9eb4 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:25.263407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157b087-591a-435d-adaf-ad06e8bc7473 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:25.263700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157b087-591a-435d-adaf-ad06e8bc7473 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:25.263978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157b087-591a-435d-adaf-ad06e8bc7473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:25.281503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b157b087-591a-435d-adaf-ad06e8bc7473 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:25.282686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b157b087-591a-435d-adaf-ad06e8bc7473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:25.345555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1165/2318] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:55:25.356259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:25.357803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:25.418700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37df67-1c9c-4504-b6a9-04c1df185f98 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:25.422288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f797ac5-838b-4953-88d1-59760214dacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:25.422942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f797ac5-838b-4953-88d1-59760214dacb service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:25.423221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f797ac5-838b-4953-88d1-59760214dacb service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:25.423522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f797ac5-838b-4953-88d1-59760214dacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:25.434096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f797ac5-838b-4953-88d1-59760214dacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:25.435667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f797ac5-838b-4953-88d1-59760214dacb service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:25.446592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1154/2319] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:55:25.898747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157b087-591a-435d-adaf-ad06e8bc7473 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:25.900393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b157b087-591a-435d-adaf-ad06e8bc7473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:25.913889 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157b087-591a-435d-adaf-ad06e8bc7473 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:25.918827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:25.919888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:25.920266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:25.920638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] PATH_INFO: `/v3/projects/14557e24f9f74819a96e5f61d0c4616b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:25.922055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14557e24f9f74819a96e5f61d0c4616b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:25.924144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:26.027818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:26.030529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14557e24f9f74819a96e5f61d0c4616b', 'request_id': 'req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.036041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14557e24f9f74819a96e5f61d0c4616b', 'request_id': 'req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.120985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14557e24f9f74819a96e5f61d0c4616b', 'request_id': 'req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.132446 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] The token cache is being invalidate because project 14557e24f9f74819a96e5f61d0c4616b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:55:26.132737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.136242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.136242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1166/2320] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:55:25 2022] DELETE /identity/v3/projects/14557e24f9f74819a96e5f61d0c4616b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:26.138880 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f797ac5-838b-4953-88d1-59760214dacb service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:26.148970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f797ac5-838b-4953-88d1-59760214dacb service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:26.158960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:26.211371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:26.217479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:26.217479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:26.217479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] PATH_INFO: `/v3/users/cb16394e90e2474eb30760b6e7e10798` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:26.217479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb16394e90e2474eb30760b6e7e10798)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:26.223012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:26.409711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb16394e90e2474eb30760b6e7e10798'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.410390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb16394e90e2474eb30760b6e7e10798'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.411003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb16394e90e2474eb30760b6e7e10798'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.418778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb16394e90e2474eb30760b6e7e10798'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.431294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b62b9f28-5430-496c-9afa-2a4f3798d052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb16394e90e2474eb30760b6e7e10798'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:26.509448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1155/2321] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:55:26 2022] DELETE /identity/v3/users/cb16394e90e2474eb30760b6e7e10798 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:26.951698 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:26.953139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:26.966352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396dd3e-e27d-4b2e-a1b5-6f18878841d8 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.027104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.028049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.028334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.028609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] PATH_INFO: `/v3/projects/6051d0cdfc1f4a7bb251b54cb4aeff20` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.029804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6051d0cdfc1f4a7bb251b54cb4aeff20)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.031800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.140210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6051d0cdfc1f4a7bb251b54cb4aeff20', 'request_id': 'req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:27.146307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6051d0cdfc1f4a7bb251b54cb4aeff20', 'request_id': 'req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:27.227971 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6051d0cdfc1f4a7bb251b54cb4aeff20', 'request_id': 'req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:27.241239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] The token cache is being invalidate because project 6051d0cdfc1f4a7bb251b54cb4aeff20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:55:27.241893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:27.242174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-028e12d2-de5b-4eb1-bc81-a0b6a2e51154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:27.243810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1167/2322] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:55:26 2022] DELETE /identity/v3/projects/6051d0cdfc1f4a7bb251b54cb4aeff20 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:27.249491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15570e-057a-4a51-9953-a5e5426c369c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.250142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15570e-057a-4a51-9953-a5e5426c369c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.250142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15570e-057a-4a51-9953-a5e5426c369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.267498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4f15570e-057a-4a51-9953-a5e5426c369c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:27.269013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4f15570e-057a-4a51-9953-a5e5426c369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:27.330539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1156/2323] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:55:27.338201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa085b-f22a-408c-9092-374ec7bae627 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.338497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa085b-f22a-408c-9092-374ec7bae627 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.338780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa085b-f22a-408c-9092-374ec7bae627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.357662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cafa085b-f22a-408c-9092-374ec7bae627 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:27.358881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cafa085b-f22a-408c-9092-374ec7bae627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:27.369451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1168/2324] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:55:27.373676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15570e-057a-4a51-9953-a5e5426c369c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.383494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15570e-057a-4a51-9953-a5e5426c369c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.388080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.388748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.389026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.389295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.390019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:27.390325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.391977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.403452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1157/2325] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:27.407578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa085b-f22a-408c-9092-374ec7bae627 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.417408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa085b-f22a-408c-9092-374ec7bae627 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.422046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5d05c-f3f6-4c24-8707-4165ed822e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.422700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5d05c-f3f6-4c24-8707-4165ed822e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.422986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5d05c-f3f6-4c24-8707-4165ed822e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.423319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5d05c-f3f6-4c24-8707-4165ed822e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.424018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5d05c-f3f6-4c24-8707-4165ed822e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.425913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5d05c-f3f6-4c24-8707-4165ed822e79 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.527687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1169/2326] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:55:27 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:27.528927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.547192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9817a-bb74-414d-9795-3e7ac83661c3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.566229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:27.622407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad544-6094-4c1b-a337-f37ddbdc6227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.623403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad544-6094-4c1b-a337-f37ddbdc6227 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.624022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad544-6094-4c1b-a337-f37ddbdc6227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.624526 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad544-6094-4c1b-a337-f37ddbdc6227 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.625464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ad544-6094-4c1b-a337-f37ddbdc6227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.628044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ad544-6094-4c1b-a337-f37ddbdc6227 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.734250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1158/2327] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:55:27 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:27.738493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5d05c-f3f6-4c24-8707-4165ed822e79 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.750658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5d05c-f3f6-4c24-8707-4165ed822e79 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.756345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdade65c-e28f-4990-9b32-7bb0dcc44b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.757110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdade65c-e28f-4990-9b32-7bb0dcc44b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.757531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdade65c-e28f-4990-9b32-7bb0dcc44b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.757877 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdade65c-e28f-4990-9b32-7bb0dcc44b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.758683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdade65c-e28f-4990-9b32-7bb0dcc44b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.760401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdade65c-e28f-4990-9b32-7bb0dcc44b17 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.782043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1170/2328] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:27.783886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad544-6094-4c1b-a337-f37ddbdc6227 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.800893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad544-6094-4c1b-a337-f37ddbdc6227 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.809780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5115bbd-1d95-4dca-b760-6e6cbab81618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.810645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5115bbd-1d95-4dca-b760-6e6cbab81618 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.811405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5115bbd-1d95-4dca-b760-6e6cbab81618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.811405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5115bbd-1d95-4dca-b760-6e6cbab81618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.812281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5115bbd-1d95-4dca-b760-6e6cbab81618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.814044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5115bbd-1d95-4dca-b760-6e6cbab81618 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.839295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1159/2329] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:27.840897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdade65c-e28f-4990-9b32-7bb0dcc44b17 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:27.852336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdade65c-e28f-4990-9b32-7bb0dcc44b17 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:27.857070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:27.857976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:27.858260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:27.858533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] PATH_INFO: `/v3/projects/81bdce0a725b4a92b4c9180df0c88300/users/be99ac4a8d2f4073a439575f18315d76/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:27.875080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bdce0a725b4a92b4c9180df0c88300, user_id=be99ac4a8d2f4073a439575f18315d76, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:27.877818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:27.990207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:27.996252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1171/2330] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:55:27 2022] PUT /identity/v3/projects/81bdce0a725b4a92b4c9180df0c88300/users/be99ac4a8d2f4073a439575f18315d76/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:28.058841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93dbb-7d8f-4461-b816-b94fbe831fb3 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.059135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93dbb-7d8f-4461-b816-b94fbe831fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.060986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a93dbb-7d8f-4461-b816-b94fbe831fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.088126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-68a93dbb-7d8f-4461-b816-b94fbe831fb3 None None] MFA Rules not processed for user `be99ac4a8d2f4073a439575f18315d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:28.089177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-68a93dbb-7d8f-4461-b816-b94fbe831fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:28.135230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.137223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:28.160496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1160/2331] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:55:28.165073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a93dbb-7d8f-4461-b816-b94fbe831fb3 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.219004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a4cc2-2223-4228-b607-6828c9e7d753 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.224210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.224940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.225265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.225585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.227441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:28.233023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a93dbb-7d8f-4461-b816-b94fbe831fb3 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.238848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf02af-9a05-4c0d-ab18-67630f748c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.239068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf02af-9a05-4c0d-ab18-67630f748c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.239191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.239367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf02af-9a05-4c0d-ab18-67630f748c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.239493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf02af-9a05-4c0d-ab18-67630f748c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.240172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf02af-9a05-4c0d-ab18-67630f748c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.240647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.242058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf02af-9a05-4c0d-ab18-67630f748c9e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.250686 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1172/2332] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:55:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:55:28.270633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.272669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:28.289441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1631b1-7afc-4551-ab38-b31be8ad02bb service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.294634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.295777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.298031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.298516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] PATH_INFO: `/v3/users/993a59e8746948ab8a0f11d877314e02` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.299865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993a59e8746948ab8a0f11d877314e02)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.301876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.368263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1161/2333] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:55:28 2022] POST /identity/v3/projects => generated 369 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:55:28.372362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf02af-9a05-4c0d-ab18-67630f748c9e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.385575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:28.433490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf02af-9a05-4c0d-ab18-67630f748c9e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.438154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.438819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.439103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.439415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.440126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.442032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.498892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993a59e8746948ab8a0f11d877314e02'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:28.499550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993a59e8746948ab8a0f11d877314e02'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:28.500174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993a59e8746948ab8a0f11d877314e02'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:28.507559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993a59e8746948ab8a0f11d877314e02'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:28.519183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993a59e8746948ab8a0f11d877314e02'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:28.531153 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1162/2334] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:55:28 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:55:28.534985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.537390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:28.550402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee12fe0-29c5-462f-a36e-7585cb8be9d1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.557965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc9a36-2dd9-43f5-a835-95ee283dc192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.558636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc9a36-2dd9-43f5-a835-95ee283dc192 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.558924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc9a36-2dd9-43f5-a835-95ee283dc192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.559208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc9a36-2dd9-43f5-a835-95ee283dc192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.563623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc9a36-2dd9-43f5-a835-95ee283dc192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.563623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc9a36-2dd9-43f5-a835-95ee283dc192 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.572662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1163/2335] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:55:28.576799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc9a36-2dd9-43f5-a835-95ee283dc192 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.586509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc9a36-2dd9-43f5-a835-95ee283dc192 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.591044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6c9ea-a459-4254-87e5-56e1a7e38364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.591915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6c9ea-a459-4254-87e5-56e1a7e38364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.592194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6c9ea-a459-4254-87e5-56e1a7e38364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.592467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6c9ea-a459-4254-87e5-56e1a7e38364 admin admin] PATH_INFO: `/v3/projects/01b2e7996eb643ba93300f8427b230f6/users/1b1e723082624a5bac2faeb4e2f51e9a/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.606985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b6c9ea-a459-4254-87e5-56e1a7e38364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b2e7996eb643ba93300f8427b230f6, user_id=1b1e723082624a5bac2faeb4e2f51e9a, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.609724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b6c9ea-a459-4254-87e5-56e1a7e38364 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.617218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1173/2336] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:55:28 2022] DELETE /identity/v3/users/993a59e8746948ab8a0f11d877314e02 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:55:28.681891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1164/2337] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:55:28 2022] PUT /identity/v3/projects/01b2e7996eb643ba93300f8427b230f6/users/1b1e723082624a5bac2faeb4e2f51e9a/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:55:28.682614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.759895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e29661-b21a-4473-a3ba-d2e4c5176f48 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.764952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:28.818051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b45621-d9cd-4483-a3f6-00bcad20b773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.818768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b45621-d9cd-4483-a3f6-00bcad20b773 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.819056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b45621-d9cd-4483-a3f6-00bcad20b773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.819382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b45621-d9cd-4483-a3f6-00bcad20b773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.820113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b45621-d9cd-4483-a3f6-00bcad20b773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.821775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b45621-d9cd-4483-a3f6-00bcad20b773 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.831754 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1174/2338] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:28.835949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6c9ea-a459-4254-87e5-56e1a7e38364 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.846279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6c9ea-a459-4254-87e5-56e1a7e38364 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.851669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.852558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.852842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.853118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] PATH_INFO: `/v3/projects/01b2e7996eb643ba93300f8427b230f6/users/1b1e723082624a5bac2faeb4e2f51e9a/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.855158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b2e7996eb643ba93300f8427b230f6, user_id=1b1e723082624a5bac2faeb4e2f51e9a, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.857918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.868741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1165/2339] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:55:28 2022] PUT /identity/v3/projects/01b2e7996eb643ba93300f8427b230f6/users/1b1e723082624a5bac2faeb4e2f51e9a/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:28.878892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2aee-b854-40a8-8a54-c3425e016621 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.879204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2aee-b854-40a8-8a54-c3425e016621 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.879519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2aee-b854-40a8-8a54-c3425e016621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.887688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:28.889432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:28.907213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-981c2aee-b854-40a8-8a54-c3425e016621 None None] MFA Rules not processed for user `1b1e723082624a5bac2faeb4e2f51e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:28.909136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-981c2aee-b854-40a8-8a54-c3425e016621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:28.964611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751790e9-cbed-4af6-bf18-d92b4eab2cc3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:28.969347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:28.970205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:28.970491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:28.970770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] PATH_INFO: `/v3/projects/876864a8f283481a81ee65a1c97d0f4f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:28.972270 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=876864a8f283481a81ee65a1c97d0f4f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:28.974198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:28.980596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1175/2340] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:28 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:55:29.075784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '876864a8f283481a81ee65a1c97d0f4f', 'request_id': 'req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:29.080469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:29.084644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '876864a8f283481a81ee65a1c97d0f4f', 'request_id': 'req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:29.177703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '876864a8f283481a81ee65a1c97d0f4f', 'request_id': 'req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:29.190940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] The token cache is being invalidate because project 876864a8f283481a81ee65a1c97d0f4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:55:29.191228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:29.191902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-9d2fe1a7-c4ba-44e6-8d35-2c4b964527d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:55:29.193432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1166/2341] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:55:28 2022] DELETE /identity/v3/projects/876864a8f283481a81ee65a1c97d0f4f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:55:29.199558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70ac3a-542a-4c07-8f6e-6f0bf1486af7 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.200072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70ac3a-542a-4c07-8f6e-6f0bf1486af7 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.200192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70ac3a-542a-4c07-8f6e-6f0bf1486af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.218307 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-bc70ac3a-542a-4c07-8f6e-6f0bf1486af7 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:29.219524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-bc70ac3a-542a-4c07-8f6e-6f0bf1486af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:29.278955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1176/2342] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:55:29.286878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a6dfb-5493-48f9-a3a4-cd453fcbedd2 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.287793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a6dfb-5493-48f9-a3a4-cd453fcbedd2 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.288077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a6dfb-5493-48f9-a3a4-cd453fcbedd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.305729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ef8a6dfb-5493-48f9-a3a4-cd453fcbedd2 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:29.306929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ef8a6dfb-5493-48f9-a3a4-cd453fcbedd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:29.314736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1167/2343] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:55:29.318868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70ac3a-542a-4c07-8f6e-6f0bf1486af7 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.329594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70ac3a-542a-4c07-8f6e-6f0bf1486af7 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.334521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.335216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.335544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.335820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.336537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:55:29.336846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.338458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.350312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1177/2344] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:55:29.354374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a6dfb-5493-48f9-a3a4-cd453fcbedd2 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.363298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a6dfb-5493-48f9-a3a4-cd453fcbedd2 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.368072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b042-6d0e-43f9-827c-4543a1caf946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.368930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b042-6d0e-43f9-827c-4543a1caf946 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.369505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b042-6d0e-43f9-827c-4543a1caf946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.369505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b042-6d0e-43f9-827c-4543a1caf946 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.370211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b042-6d0e-43f9-827c-4543a1caf946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.372122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b042-6d0e-43f9-827c-4543a1caf946 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.468505 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1168/2345] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:55:29 2022] POST /identity/v3/projects => generated 355 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:55:29.472547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.482602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f0121-690e-4f51-8962-425190a77752 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.494590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:29.537776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e369-d079-48a5-9b10-e60a05ad9387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.538449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e369-d079-48a5-9b10-e60a05ad9387 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.538733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e369-d079-48a5-9b10-e60a05ad9387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.539010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5e369-d079-48a5-9b10-e60a05ad9387 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.539743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5e369-d079-48a5-9b10-e60a05ad9387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.541662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5e369-d079-48a5-9b10-e60a05ad9387 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.611966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1178/2346] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:55:29 2022] POST /identity/v3/users => generated 278 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:29.615961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b042-6d0e-43f9-827c-4543a1caf946 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.626360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b042-6d0e-43f9-827c-4543a1caf946 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.631472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7db152-155c-40e4-b581-32988be33fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.632128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7db152-155c-40e4-b581-32988be33fef admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.632411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7db152-155c-40e4-b581-32988be33fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.632689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7db152-155c-40e4-b581-32988be33fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.633417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7db152-155c-40e4-b581-32988be33fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.634994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7db152-155c-40e4-b581-32988be33fef admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.645234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1169/2347] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:29.649578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e369-d079-48a5-9b10-e60a05ad9387 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.659522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5e369-d079-48a5-9b10-e60a05ad9387 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.664087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a576610-4dd8-4388-8213-7a6a10431a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.664746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a576610-4dd8-4388-8213-7a6a10431a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.665033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a576610-4dd8-4388-8213-7a6a10431a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.665313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a576610-4dd8-4388-8213-7a6a10431a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.666057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a576610-4dd8-4388-8213-7a6a10431a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.667667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a576610-4dd8-4388-8213-7a6a10431a63 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.677358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1179/2348] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:29.681265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7db152-155c-40e4-b581-32988be33fef admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.691025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7db152-155c-40e4-b581-32988be33fef admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.696086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f12b0-5e8d-40a4-82b6-b360d2ef88c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.697208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f12b0-5e8d-40a4-82b6-b360d2ef88c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.697493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f12b0-5e8d-40a4-82b6-b360d2ef88c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.697770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f12b0-5e8d-40a4-82b6-b360d2ef88c0 admin admin] PATH_INFO: `/v3/projects/09ca5219beb44efea7223c9447bfe55c/users/f66465bdef7645b0a795e68a7cb0ec2b/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.712291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f12b0-5e8d-40a4-82b6-b360d2ef88c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ca5219beb44efea7223c9447bfe55c, user_id=f66465bdef7645b0a795e68a7cb0ec2b, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.714917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f12b0-5e8d-40a4-82b6-b360d2ef88c0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:29.781709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1170/2349] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:55:29 2022] PUT /identity/v3/projects/09ca5219beb44efea7223c9447bfe55c/users/f66465bdef7645b0a795e68a7cb0ec2b/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:29.792181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27bee4-484c-4489-9f94-72c03da04651 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.792471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27bee4-484c-4489-9f94-72c03da04651 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.792750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27bee4-484c-4489-9f94-72c03da04651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.819589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9d27bee4-484c-4489-9f94-72c03da04651 None None] MFA Rules not processed for user `f66465bdef7645b0a795e68a7cb0ec2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:29.820835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9d27bee4-484c-4489-9f94-72c03da04651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:29.883448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1180/2350] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:29 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:55:29.887770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f12b0-5e8d-40a4-82b6-b360d2ef88c0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:29.948463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f12b0-5e8d-40a4-82b6-b360d2ef88c0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:29.953249 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd947-0863-4537-a38e-00a71a6ee1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:29.953907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd947-0863-4537-a38e-00a71a6ee1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:29.954188 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd947-0863-4537-a38e-00a71a6ee1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:29.954461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd947-0863-4537-a38e-00a71a6ee1f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:29.955150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd947-0863-4537-a38e-00a71a6ee1f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:29.957076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd947-0863-4537-a38e-00a71a6ee1f9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.046412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1171/2351] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:55:29 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:30.050600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27bee4-484c-4489-9f94-72c03da04651 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.061932 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27bee4-484c-4489-9f94-72c03da04651 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.118112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d259260-4755-4b9d-b19a-7b2be2931e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.118814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d259260-4755-4b9d-b19a-7b2be2931e5a admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.119092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d259260-4755-4b9d-b19a-7b2be2931e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.119402 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d259260-4755-4b9d-b19a-7b2be2931e5a admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.120087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d259260-4755-4b9d-b19a-7b2be2931e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.122005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d259260-4755-4b9d-b19a-7b2be2931e5a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.212015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1181/2352] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:55:30 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:55:30.215975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd947-0863-4537-a38e-00a71a6ee1f9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.226753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd947-0863-4537-a38e-00a71a6ee1f9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.231386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868a2ee-1322-4f7e-b04d-287ef6ce3b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.232037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a2ee-1322-4f7e-b04d-287ef6ce3b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.232322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a2ee-1322-4f7e-b04d-287ef6ce3b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.232602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868a2ee-1322-4f7e-b04d-287ef6ce3b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.233330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868a2ee-1322-4f7e-b04d-287ef6ce3b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.234915 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868a2ee-1322-4f7e-b04d-287ef6ce3b59 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.245961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1172/2353] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:55:30.249929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d259260-4755-4b9d-b19a-7b2be2931e5a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.259835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d259260-4755-4b9d-b19a-7b2be2931e5a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.264336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4d426-9997-4a2f-86dd-a47d88c61f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.265206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4d426-9997-4a2f-86dd-a47d88c61f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.265488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4d426-9997-4a2f-86dd-a47d88c61f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.265763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4d426-9997-4a2f-86dd-a47d88c61f4f admin admin] PATH_INFO: `/v3/projects/794ddf8b808747d79588dede429b007d/users/1309b6e99eb04aa682fdf0dc2f839c00/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.277734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:30.280690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4d426-9997-4a2f-86dd-a47d88c61f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=794ddf8b808747d79588dede429b007d, user_id=1309b6e99eb04aa682fdf0dc2f839c00, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.283347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4d426-9997-4a2f-86dd-a47d88c61f4f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.355374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1182/2354] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:55:30 2022] PUT /identity/v3/projects/794ddf8b808747d79588dede429b007d/users/1309b6e99eb04aa682fdf0dc2f839c00/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:30.359262 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868a2ee-1322-4f7e-b04d-287ef6ce3b59 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.369695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868a2ee-1322-4f7e-b04d-287ef6ce3b59 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.417354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:30.423592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fc83b-5421-4440-aa09-a678596a4103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.428282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fc83b-5421-4440-aa09-a678596a4103 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.428282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fc83b-5421-4440-aa09-a678596a4103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.428282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fc83b-5421-4440-aa09-a678596a4103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.428282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fc83b-5421-4440-aa09-a678596a4103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.428282 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fc83b-5421-4440-aa09-a678596a4103 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.437235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1173/2355] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:55:30.441278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4d426-9997-4a2f-86dd-a47d88c61f4f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:55:30.454274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4d426-9997-4a2f-86dd-a47d88c61f4f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:55:30.460046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:55:30.460345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.460481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.461760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] PATH_INFO: `/v3/projects/794ddf8b808747d79588dede429b007d/users/1309b6e99eb04aa682fdf0dc2f839c00/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.466344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=794ddf8b808747d79588dede429b007d, user_id=1309b6e99eb04aa682fdf0dc2f839c00, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:55:30.468984 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:55:30.480871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1183/2356] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:55:30 2022] PUT /identity/v3/projects/794ddf8b808747d79588dede429b007d/users/1309b6e99eb04aa682fdf0dc2f839c00/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:55:30.491696 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525692ec-867b-4337-b348-64047e1eb125 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:55:30.496936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525692ec-867b-4337-b348-64047e1eb125 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:55:30.496936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525692ec-867b-4337-b348-64047e1eb125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:55:30.518035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-525692ec-867b-4337-b348-64047e1eb125 None None] MFA Rules not processed for user `1309b6e99eb04aa682fdf0dc2f839c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:55:30.519289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-525692ec-867b-4337-b348-64047e1eb125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:55:30.590756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1174/2357] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:55:30 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:55:40.477072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=8,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:55:40.587866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=38 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:29.050911 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:29.053019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:29.120991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce902570-b799-4ad2-93e2-aeba73018851 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:29.125488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:29.126351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:29.126634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:29.126907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] PATH_INFO: `/v3/users/be99ac4a8d2f4073a439575f18315d76` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:29.128208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] RBAC: Authorizing `identity:delete_user(user_id=be99ac4a8d2f4073a439575f18315d76)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:29.130135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:29.269506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be99ac4a8d2f4073a439575f18315d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:29.270154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be99ac4a8d2f4073a439575f18315d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:29.270794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be99ac4a8d2f4073a439575f18315d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:29.277717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be99ac4a8d2f4073a439575f18315d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:29.289288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be99ac4a8d2f4073a439575f18315d76'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:29.346635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1184/2358] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:56:29 2022] DELETE /identity/v3/users/be99ac4a8d2f4073a439575f18315d76 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:29.351527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945082c-96f2-49d1-b009-ef918515db01 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:29.355244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945082c-96f2-49d1-b009-ef918515db01 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:29.355244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945082c-96f2-49d1-b009-ef918515db01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:29.369588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a945082c-96f2-49d1-b009-ef918515db01 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:29.370805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a945082c-96f2-49d1-b009-ef918515db01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:29.427011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1175/2359] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:56:29.437895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:29.441191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:29.524308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab399e7-5dd1-4da6-be50-289e8702861e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:29.527981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611fd9-d27b-4b58-9e27-4d37d1847ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:29.528623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611fd9-d27b-4b58-9e27-4d37d1847ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:29.528905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611fd9-d27b-4b58-9e27-4d37d1847ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:29.529183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611fd9-d27b-4b58-9e27-4d37d1847ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:29.535268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:29.539447 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25611fd9-d27b-4b58-9e27-4d37d1847ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:29.540863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25611fd9-d27b-4b58-9e27-4d37d1847ad6 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:29.550118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1185/2360] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:56:29.931983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945082c-96f2-49d1-b009-ef918515db01 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:29.933709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a945082c-96f2-49d1-b009-ef918515db01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:29.945939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945082c-96f2-49d1-b009-ef918515db01 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:29.950834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:29.951863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:29.952149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:29.952430 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] PATH_INFO: `/v3/projects/81bdce0a725b4a92b4c9180df0c88300` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:29.953661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bdce0a725b4a92b4c9180df0c88300)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:29.955613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:30.055242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bdce0a725b4a92b4c9180df0c88300', 'request_id': 'req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.060943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bdce0a725b4a92b4c9180df0c88300', 'request_id': 'req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.134358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bdce0a725b4a92b4c9180df0c88300', 'request_id': 'req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.145877 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] The token cache is being invalidate because project 81bdce0a725b4a92b4c9180df0c88300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:56:30.146173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.146803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.149063 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1176/2361] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:56:29 2022] DELETE /identity/v3/projects/81bdce0a725b4a92b4c9180df0c88300 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:30.153045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611fd9-d27b-4b58-9e27-4d37d1847ad6 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:30.162549 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25611fd9-d27b-4b58-9e27-4d37d1847ad6 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:30.217610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:30.218414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:30.218730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:30.219011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] PATH_INFO: `/v3/users/1b1e723082624a5bac2faeb4e2f51e9a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:30.220241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b1e723082624a5bac2faeb4e2f51e9a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:30.222367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:30.383342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b1e723082624a5bac2faeb4e2f51e9a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.383998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b1e723082624a5bac2faeb4e2f51e9a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.384604 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b1e723082624a5bac2faeb4e2f51e9a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.391406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b1e723082624a5bac2faeb4e2f51e9a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.403566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-74c03fc8-6202-42dd-b8c0-188cd9534742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b1e723082624a5bac2faeb4e2f51e9a'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:30.500860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1186/2362] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:56:30 2022] DELETE /identity/v3/users/1b1e723082624a5bac2faeb4e2f51e9a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:30.902727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:30.904204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:30.917745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629017a-9b43-4c9e-ba2e-4b9254ed5e0e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:30.981823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:30.982707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:30.982991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:30.983294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] PATH_INFO: `/v3/projects/01b2e7996eb643ba93300f8427b230f6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:30.984496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b2e7996eb643ba93300f8427b230f6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:30.986382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:31.085662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:31.101556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b2e7996eb643ba93300f8427b230f6', 'request_id': 'req-fa9193db-42c2-46e3-a523-75b5c7cc4926'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:31.106897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b2e7996eb643ba93300f8427b230f6', 'request_id': 'req-fa9193db-42c2-46e3-a523-75b5c7cc4926'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:31.180592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b2e7996eb643ba93300f8427b230f6', 'request_id': 'req-fa9193db-42c2-46e3-a523-75b5c7cc4926'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:31.191790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] The token cache is being invalidate because project 01b2e7996eb643ba93300f8427b230f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:56:31.192583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:31.192839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fa9193db-42c2-46e3-a523-75b5c7cc4926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:56:31.194247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1177/2363] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:56:30 2022] DELETE /identity/v3/projects/01b2e7996eb643ba93300f8427b230f6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:31.201033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135075b-215c-4d44-b6a4-d1f13531342a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.201519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135075b-215c-4d44-b6a4-d1f13531342a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.201629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4135075b-215c-4d44-b6a4-d1f13531342a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.219135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4135075b-215c-4d44-b6a4-d1f13531342a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:31.221349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4135075b-215c-4d44-b6a4-d1f13531342a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:31.278010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1187/2364] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:56:31.285980 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262f248-701b-466c-9db8-0b599b3e7305 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.286269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262f248-701b-466c-9db8-0b599b3e7305 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.286576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262f248-701b-466c-9db8-0b599b3e7305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.304738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4262f248-701b-466c-9db8-0b599b3e7305 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:31.305928 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4262f248-701b-466c-9db8-0b599b3e7305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:31.313959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1178/2365] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:56:31.318076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135075b-215c-4d44-b6a4-d1f13531342a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:31.327860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4135075b-215c-4d44-b6a4-d1f13531342a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:31.332324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:31.332968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.333255 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.333532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.334250 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:56:31.335075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:31.336690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:31.345024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:31.348180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1188/2366] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:56:31.352944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262f248-701b-466c-9db8-0b599b3e7305 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:31.361981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262f248-701b-466c-9db8-0b599b3e7305 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:31.366774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7774e7-910f-4372-8822-a32697147b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:31.367439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7774e7-910f-4372-8822-a32697147b43 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.367716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7774e7-910f-4372-8822-a32697147b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.368033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7774e7-910f-4372-8822-a32697147b43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.368723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7774e7-910f-4372-8822-a32697147b43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:31.370604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7774e7-910f-4372-8822-a32697147b43 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:31.480521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1179/2367] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:56:31 2022] POST /identity/v3/projects => generated 409 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:56:31.484640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:31.494566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b65f0-af7d-45ee-8665-8b73a9d678fa admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:31.549763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242c531-bbf6-4a19-9d89-018d7bc5a15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:31.550436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c531-bbf6-4a19-9d89-018d7bc5a15b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.550720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c531-bbf6-4a19-9d89-018d7bc5a15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.550999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c531-bbf6-4a19-9d89-018d7bc5a15b admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.551727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242c531-bbf6-4a19-9d89-018d7bc5a15b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:31.553669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242c531-bbf6-4a19-9d89-018d7bc5a15b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:31.632745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1189/2368] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:56:31 2022] POST /identity/v3/users => generated 305 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:56:31.638200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7774e7-910f-4372-8822-a32697147b43 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:31.649038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7774e7-910f-4372-8822-a32697147b43 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:31.654592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021c31e-2f08-43ac-89c6-4e1460e26eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:31.654592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c31e-2f08-43ac-89c6-4e1460e26eae admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.654592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c31e-2f08-43ac-89c6-4e1460e26eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.654936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021c31e-2f08-43ac-89c6-4e1460e26eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.655513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021c31e-2f08-43ac-89c6-4e1460e26eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:31.657099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021c31e-2f08-43ac-89c6-4e1460e26eae admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:31.668002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1180/2369] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:56:31.671939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242c531-bbf6-4a19-9d89-018d7bc5a15b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:31.682407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242c531-bbf6-4a19-9d89-018d7bc5a15b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:31.687170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50cfa-3d08-4705-b91c-9f782c1d35f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:31.687882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50cfa-3d08-4705-b91c-9f782c1d35f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.688171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50cfa-3d08-4705-b91c-9f782c1d35f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.688450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50cfa-3d08-4705-b91c-9f782c1d35f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.689180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce50cfa-3d08-4705-b91c-9f782c1d35f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:31.690776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce50cfa-3d08-4705-b91c-9f782c1d35f1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:31.703169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1190/2370] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:56:31.706407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021c31e-2f08-43ac-89c6-4e1460e26eae admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:31.717360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021c31e-2f08-43ac-89c6-4e1460e26eae admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:31.722028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67c26c-5830-45b1-b645-2010c60dcc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:31.723768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67c26c-5830-45b1-b645-2010c60dcc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.724050 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67c26c-5830-45b1-b645-2010c60dcc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.724325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67c26c-5830-45b1-b645-2010c60dcc67 admin admin] PATH_INFO: `/v3/projects/fdbcae1524b644659927d3d6bb80a9b0/users/55243615975f4da385cbd7e1a7fe45cd/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.739525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67c26c-5830-45b1-b645-2010c60dcc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbcae1524b644659927d3d6bb80a9b0, user_id=55243615975f4da385cbd7e1a7fe45cd, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:31.742227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67c26c-5830-45b1-b645-2010c60dcc67 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:31.804606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1181/2371] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:56:31 2022] PUT /identity/v3/projects/fdbcae1524b644659927d3d6bb80a9b0/users/55243615975f4da385cbd7e1a7fe45cd/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:31.814688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31540195-cbd6-45f5-926e-be3e372f937c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.814983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31540195-cbd6-45f5-926e-be3e372f937c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.815294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31540195-cbd6-45f5-926e-be3e372f937c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.843481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-31540195-cbd6-45f5-926e-be3e372f937c None None] MFA Rules not processed for user `55243615975f4da385cbd7e1a7fe45cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:31.844697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-31540195-cbd6-45f5-926e-be3e372f937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:31.911711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1191/2372] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:56:31 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:56:31.916265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67c26c-5830-45b1-b645-2010c60dcc67 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:31.977786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67c26c-5830-45b1-b645-2010c60dcc67 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:31.982373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd96ed5-26b5-4ba7-8872-5d4e03ff93a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:31.983017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd96ed5-26b5-4ba7-8872-5d4e03ff93a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:31.983322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd96ed5-26b5-4ba7-8872-5d4e03ff93a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:31.983605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd96ed5-26b5-4ba7-8872-5d4e03ff93a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:31.984293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd96ed5-26b5-4ba7-8872-5d4e03ff93a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:31.986169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd96ed5-26b5-4ba7-8872-5d4e03ff93a2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:32.056884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1182/2373] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:56:31 2022] POST /identity/v3/projects => generated 409 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:56:32.061170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31540195-cbd6-45f5-926e-be3e372f937c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:32.073219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31540195-cbd6-45f5-926e-be3e372f937c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:32.127860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef06c4b-febd-44f0-94bd-e598d6186bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:32.128526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef06c4b-febd-44f0-94bd-e598d6186bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:32.128813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef06c4b-febd-44f0-94bd-e598d6186bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:32.129091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef06c4b-febd-44f0-94bd-e598d6186bc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:32.129770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef06c4b-febd-44f0-94bd-e598d6186bc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:32.132329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef06c4b-febd-44f0-94bd-e598d6186bc0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:32.227351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:32.233595 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1192/2374] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:56:32 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:56:32.237569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd96ed5-26b5-4ba7-8872-5d4e03ff93a2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:32.248586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd96ed5-26b5-4ba7-8872-5d4e03ff93a2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:32.253167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47171edb-a030-4870-9475-3f42ba07c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:32.253813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47171edb-a030-4870-9475-3f42ba07c1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:32.254099 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47171edb-a030-4870-9475-3f42ba07c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:32.254375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47171edb-a030-4870-9475-3f42ba07c1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:32.255100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47171edb-a030-4870-9475-3f42ba07c1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:32.257428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47171edb-a030-4870-9475-3f42ba07c1fe admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:32.267789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1183/2375] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:56:32.271827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef06c4b-febd-44f0-94bd-e598d6186bc0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:32.281811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef06c4b-febd-44f0-94bd-e598d6186bc0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:32.286388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccb838-b816-4ea8-99e4-55dd75ec0d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:32.287284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccb838-b816-4ea8-99e4-55dd75ec0d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:32.287583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccb838-b816-4ea8-99e4-55dd75ec0d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:32.287863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ccb838-b816-4ea8-99e4-55dd75ec0d47 admin admin] PATH_INFO: `/v3/projects/12cadb512244446e94b144b03b673ff6/users/1c43a11942724ea6bd886d602b1da9b9/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:32.302510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccb838-b816-4ea8-99e4-55dd75ec0d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12cadb512244446e94b144b03b673ff6, user_id=1c43a11942724ea6bd886d602b1da9b9, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:32.305674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ccb838-b816-4ea8-99e4-55dd75ec0d47 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:32.380758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1193/2376] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:56:32 2022] PUT /identity/v3/projects/12cadb512244446e94b144b03b673ff6/users/1c43a11942724ea6bd886d602b1da9b9/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:32.384657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47171edb-a030-4870-9475-3f42ba07c1fe admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:32.394447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47171edb-a030-4870-9475-3f42ba07c1fe admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:32.414508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:32.450388 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f687a1-0cbf-4011-aad6-1352eadcca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:32.451046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f687a1-0cbf-4011-aad6-1352eadcca7f admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:32.451362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f687a1-0cbf-4011-aad6-1352eadcca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:32.451641 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f687a1-0cbf-4011-aad6-1352eadcca7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:32.452358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f687a1-0cbf-4011-aad6-1352eadcca7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:32.453942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f687a1-0cbf-4011-aad6-1352eadcca7f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:32.463452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1184/2377] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:56:32.467693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccb838-b816-4ea8-99e4-55dd75ec0d47 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:56:32.478321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ccb838-b816-4ea8-99e4-55dd75ec0d47 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:56:32.482779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:56:32.483701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:32.483987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:32.484260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] PATH_INFO: `/v3/projects/12cadb512244446e94b144b03b673ff6/users/1c43a11942724ea6bd886d602b1da9b9/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:32.486171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=12cadb512244446e94b144b03b673ff6, user_id=1c43a11942724ea6bd886d602b1da9b9, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:56:32.488862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:56:32.499331 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1194/2378] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:56:32 2022] PUT /identity/v3/projects/12cadb512244446e94b144b03b673ff6/users/1c43a11942724ea6bd886d602b1da9b9/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:56:32.509190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656c7f0-c03f-46e2-9d5c-9d6945817fb3 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:56:32.509482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656c7f0-c03f-46e2-9d5c-9d6945817fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:56:32.509761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656c7f0-c03f-46e2-9d5c-9d6945817fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:56:32.535503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-a656c7f0-c03f-46e2-9d5c-9d6945817fb3 None None] MFA Rules not processed for user `1c43a11942724ea6bd886d602b1da9b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:56:32.536676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a656c7f0-c03f-46e2-9d5c-9d6945817fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:56:32.602925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1185/2379] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:56:32 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:56:42.495653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=16,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:56:42.599450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=52 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:24.322306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:24.324153 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:24.391720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfbbdf-a9bd-4cb1-acb6-a29394a9bb8b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:24.395326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:24.395974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:24.396259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:24.396536 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:24.398357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:24.470723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:24.471970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:24.489835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1195/2380] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:57:24.571196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656c7f0-c03f-46e2-9d5c-9d6945817fb3 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:24.580802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656c7f0-c03f-46e2-9d5c-9d6945817fb3 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:24.584248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:24.584878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:24.585156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:24.585431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:24.587751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:24.653521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:24.654945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:24.674031 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1186/2381] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:57:24.721634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:24.729994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e6018-de51-43da-8467-153ad913b1b5 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:24.733602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2d828-7f46-4a40-bf2c-8a5380ab769b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:24.734239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2d828-7f46-4a40-bf2c-8a5380ab769b service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:24.734526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2d828-7f46-4a40-bf2c-8a5380ab769b service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:24.734804 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2d828-7f46-4a40-bf2c-8a5380ab769b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:24.736596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ebb2d828-7f46-4a40-bf2c-8a5380ab769b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application [None req-ebb2d828-7f46-4a40-bf2c-8a5380ab769b service cinder] Could not find project: 01b2e7996eb643ba93300f8427b230f6.: keystone.exception.ProjectNotFound: Could not find project: 01b2e7996eb643ba93300f8427b230f6. Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 11 11:57:24.749373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:24.752462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 11:57:24.753819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application with Lock( Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._enter() Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.creator() Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application created_value = creator( Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 11:57:24.755344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 11:57:24.756820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 01b2e7996eb643ba93300f8427b230f6. Jul 11 11:57:24.756820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: ERROR keystone.server.flask.application Jul 11 11:57:24.756820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1196/2382] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:57:24 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 29 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 11 11:57:24.788867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:24.797539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced5a1d-8198-4a18-8dad-e6035318bbf9 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:24.801197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:24.801830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:24.802114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:24.802391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:24.804180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:24.867515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:24.868868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:24.878661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1187/2383] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:57:25.029233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2d828-7f46-4a40-bf2c-8a5380ab769b service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:25.038240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2d828-7f46-4a40-bf2c-8a5380ab769b service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:25.042069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:25.042823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:25.043243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:25.043643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:25.056362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:25.061278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:25.086009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1197/2384] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:57:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:57:25.138971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:25.140610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:25.197960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2277f6-8398-476f-ba83-034948f18b08 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:25.201544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:25.202175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:25.202476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:25.202752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:25.204539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:25.233842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:25.266086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:25.267546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:25.277305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1188/2385] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:57:28.774264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:28.782592 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:28.800477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc1e7e-0ff4-48d8-bb06-b8a15fb33c4a service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:28.806116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:28.807015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:28.807329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:28.807609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] PATH_INFO: `/v3/users/15ff2cacb5c1450482597a2f5fa3c8a9` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:28.808792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ff2cacb5c1450482597a2f5fa3c8a9)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:28.810708 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:28.963856 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ff2cacb5c1450482597a2f5fa3c8a9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:28.964510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ff2cacb5c1450482597a2f5fa3c8a9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:28.965110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ff2cacb5c1450482597a2f5fa3c8a9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:28.972154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ff2cacb5c1450482597a2f5fa3c8a9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:28.977080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:28.987957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0414f803-cba0-4756-a0bb-14c4afcf330d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ff2cacb5c1450482597a2f5fa3c8a9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:29.052837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1198/2386] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:57:28 2022] DELETE /identity/v3/users/15ff2cacb5c1450482597a2f5fa3c8a9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:57:29.271653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:29.282340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84e736-6c1a-4dd5-8a1d-a1e12eff0d2e service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:29.348866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:29.349718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:29.349999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:29.350268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] PATH_INFO: `/v3/projects/5779b0ffa2244901a92adbee65b10421` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:29.352053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5779b0ffa2244901a92adbee65b10421)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:29.353967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:29.429778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5779b0ffa2244901a92adbee65b10421', 'request_id': 'req-e7601f04-62ed-442a-902d-de28bd6a9961'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:29.435364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5779b0ffa2244901a92adbee65b10421', 'request_id': 'req-e7601f04-62ed-442a-902d-de28bd6a9961'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:29.498865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5779b0ffa2244901a92adbee65b10421', 'request_id': 'req-e7601f04-62ed-442a-902d-de28bd6a9961'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:29.509925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] The token cache is being invalidate because project 5779b0ffa2244901a92adbee65b10421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:57:29.513293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:29.513293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-e7601f04-62ed-442a-902d-de28bd6a9961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:57:29.513537 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1189/2387] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:57:29 2022] DELETE /identity/v3/projects/5779b0ffa2244901a92adbee65b10421 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:57:29.519487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef737b-cb33-48b0-ba7a-91cfd59fe592 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:29.519786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef737b-cb33-48b0-ba7a-91cfd59fe592 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:29.520070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef737b-cb33-48b0-ba7a-91cfd59fe592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:29.538559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f0ef737b-cb33-48b0-ba7a-91cfd59fe592 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:29.540440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f0ef737b-cb33-48b0-ba7a-91cfd59fe592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:29.597448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1199/2388] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:57:29.605748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483331eb-e642-4318-af9f-e4bc13a77a5e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:29.606037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483331eb-e642-4318-af9f-e4bc13a77a5e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:29.606315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483331eb-e642-4318-af9f-e4bc13a77a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:29.623922 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-483331eb-e642-4318-af9f-e4bc13a77a5e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:29.625112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-483331eb-e642-4318-af9f-e4bc13a77a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:29.632855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1190/2389] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:57:29.636755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef737b-cb33-48b0-ba7a-91cfd59fe592 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:29.646272 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef737b-cb33-48b0-ba7a-91cfd59fe592 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:29.651390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:29.652037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:29.652345 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:29.652606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:29.653361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:57:29.653675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:29.655301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:29.667238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1200/2390] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:57:29.671176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483331eb-e642-4318-af9f-e4bc13a77a5e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:29.680387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483331eb-e642-4318-af9f-e4bc13a77a5e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:29.684893 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a46e27-4267-4376-be62-ffc628200dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:29.685522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46e27-4267-4376-be62-ffc628200dac admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:29.685802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46e27-4267-4376-be62-ffc628200dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:29.686072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46e27-4267-4376-be62-ffc628200dac admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:29.686756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a46e27-4267-4376-be62-ffc628200dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:29.688656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a46e27-4267-4376-be62-ffc628200dac admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:29.786072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1191/2391] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:57:29 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:57:29.789930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:29.799753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6e75f-cb7c-4073-ad74-10f9fc40156c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:29.868418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67070-9423-4dc0-9a9d-dc05cdaa08f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:29.869084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67070-9423-4dc0-9a9d-dc05cdaa08f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:29.869409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67070-9423-4dc0-9a9d-dc05cdaa08f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:29.869689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b67070-9423-4dc0-9a9d-dc05cdaa08f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:29.870384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b67070-9423-4dc0-9a9d-dc05cdaa08f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:29.872979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b67070-9423-4dc0-9a9d-dc05cdaa08f1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:29.968369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1201/2392] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:57:29 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:57:29.972573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a46e27-4267-4376-be62-ffc628200dac admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:29.982942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a46e27-4267-4376-be62-ffc628200dac admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:29.987767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8f69b-4aae-448e-bafe-5d13e7652734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:29.988470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8f69b-4aae-448e-bafe-5d13e7652734 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:29.988754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8f69b-4aae-448e-bafe-5d13e7652734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:29.989030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8f69b-4aae-448e-bafe-5d13e7652734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:29.989765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8f69b-4aae-448e-bafe-5d13e7652734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:29.991381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8f69b-4aae-448e-bafe-5d13e7652734 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:30.001771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1192/2393] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:57:30.005576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67070-9423-4dc0-9a9d-dc05cdaa08f1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:30.016065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b67070-9423-4dc0-9a9d-dc05cdaa08f1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:30.020682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76f1d9-679c-47be-80b4-11a10e98ae61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:30.021340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76f1d9-679c-47be-80b4-11a10e98ae61 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:30.021626 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76f1d9-679c-47be-80b4-11a10e98ae61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:30.021926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76f1d9-679c-47be-80b4-11a10e98ae61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:30.022631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76f1d9-679c-47be-80b4-11a10e98ae61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:30.024885 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76f1d9-679c-47be-80b4-11a10e98ae61 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:30.034590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1202/2394] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:57:30.040414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8f69b-4aae-448e-bafe-5d13e7652734 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:30.050317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8f69b-4aae-448e-bafe-5d13e7652734 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:30.055590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:30.056470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:30.056782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:30.057075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] PATH_INFO: `/v3/projects/9a8692bb55544ced870a7b4f1f2c4dcf/users/d717221a65f1409d83f10ddfa374ebeb/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:30.069350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:30.074431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a8692bb55544ced870a7b4f1f2c4dcf, user_id=d717221a65f1409d83f10ddfa374ebeb, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:30.083612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:30.134089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1193/2395] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:57:30 2022] PUT /identity/v3/projects/9a8692bb55544ced870a7b4f1f2c4dcf/users/d717221a65f1409d83f10ddfa374ebeb/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:57:30.144762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66aa42-747d-4ec1-a019-38f4f1f4bb1a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:30.145058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66aa42-747d-4ec1-a019-38f4f1f4bb1a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:30.145339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66aa42-747d-4ec1-a019-38f4f1f4bb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:30.171609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4a66aa42-747d-4ec1-a019-38f4f1f4bb1a None None] MFA Rules not processed for user `d717221a65f1409d83f10ddfa374ebeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:57:30.173682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4a66aa42-747d-4ec1-a019-38f4f1f4bb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:30.211147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:30.242479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1203/2396] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:57:30.254276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:57:30.256098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:57:30.344495 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978dd3d-5441-42b1-b4ec-72fefb8625d2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:57:30.350117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247b43d-a908-42b3-a7a2-44b8b131230b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:57:30.350755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247b43d-a908-42b3-a7a2-44b8b131230b service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:57:30.351041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247b43d-a908-42b3-a7a2-44b8b131230b service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:57:30.351353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247b43d-a908-42b3-a7a2-44b8b131230b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:57:30.358995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247b43d-a908-42b3-a7a2-44b8b131230b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:57:30.360606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247b43d-a908-42b3-a7a2-44b8b131230b service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:57:30.369800 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1194/2397] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:57:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:57:40.322659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=12 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:57:40.365276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=44,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:07.772478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66aa42-747d-4ec1-a019-38f4f1f4bb1a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:07.774471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4a66aa42-747d-4ec1-a019-38f4f1f4bb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:07.844142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66aa42-747d-4ec1-a019-38f4f1f4bb1a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:07.847747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:07.848392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:07.848679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:07.848962 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:07.851221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:07.916831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:07.918347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:07.927814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1204/2398] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:07.974124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247b43d-a908-42b3-a7a2-44b8b131230b service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:07.984886 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247b43d-a908-42b3-a7a2-44b8b131230b service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:07.988936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:07.989568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:07.989847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:07.990192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:07.991984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:08.006175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:08.007924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:08.021510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1195/2399] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:08.165835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:08.170643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:08.178646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219e87c-dfd6-414b-a76d-9437e4fb20f7 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:08.186582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:08.187363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:08.187776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:08.188091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:08.190253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:08.191238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:08.193914 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:08.204922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:08.206458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:08.218117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1205/2400] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:08.218917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:08.228417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670d0c6-843c-44f1-9aa7-d12d1a85a5bb service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:08.232747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcd942-4737-46df-8e14-d2f0ad26a5d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:08.233399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd942-4737-46df-8e14-d2f0ad26a5d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:08.233682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd942-4737-46df-8e14-d2f0ad26a5d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:08.233961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd942-4737-46df-8e14-d2f0ad26a5d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:08.235907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3cfcd942-4737-46df-8e14-d2f0ad26a5d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:08.278477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893a8ec-a73e-4c0b-af2a-7494813fba13 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:08.285243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:08.286418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:08.286802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:08.288872 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] PATH_INFO: `/v3/users/f66465bdef7645b0a795e68a7cb0ec2b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:08.290969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66465bdef7645b0a795e68a7cb0ec2b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:08.293173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:08.316524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcd942-4737-46df-8e14-d2f0ad26a5d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:08.318058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcd942-4737-46df-8e14-d2f0ad26a5d1 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:08.328678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1206/2401] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:08.514673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66465bdef7645b0a795e68a7cb0ec2b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:08.515348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66465bdef7645b0a795e68a7cb0ec2b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:08.515952 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66465bdef7645b0a795e68a7cb0ec2b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:08.522757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66465bdef7645b0a795e68a7cb0ec2b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:08.538113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66465bdef7645b0a795e68a7cb0ec2b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:08.633470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1196/2402] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:08 2022] DELETE /identity/v3/users/f66465bdef7645b0a795e68a7cb0ec2b => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:08.638970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169af40-60e3-40bc-b5f4-e4d01462daf8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:08.639284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169af40-60e3-40bc-b5f4-e4d01462daf8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:08.639575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169af40-60e3-40bc-b5f4-e4d01462daf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:08.664114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1169af40-60e3-40bc-b5f4-e4d01462daf8 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:08.665682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1169af40-60e3-40bc-b5f4-e4d01462daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:08.694969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:08.729775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1207/2403] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:08.739973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:08.741416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:08.857527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169af40-60e3-40bc-b5f4-e4d01462daf8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:08.859051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1169af40-60e3-40bc-b5f4-e4d01462daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:08.873353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169af40-60e3-40bc-b5f4-e4d01462daf8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:08.878152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:08.881605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:08.881605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:08.881605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] PATH_INFO: `/v3/users/d717221a65f1409d83f10ddfa374ebeb` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:08.881605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d717221a65f1409d83f10ddfa374ebeb)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:08.882051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c7595-2d52-4aeb-8fb0-a15f881deda1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:08.884289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:08.886125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c15368-07c4-46d9-b8b5-09c46fd53069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:08.886783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15368-07c4-46d9-b8b5-09c46fd53069 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:08.887067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15368-07c4-46d9-b8b5-09c46fd53069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:08.887376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c15368-07c4-46d9-b8b5-09c46fd53069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:08.898242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c15368-07c4-46d9-b8b5-09c46fd53069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:08.900608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c15368-07c4-46d9-b8b5-09c46fd53069 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:08.906244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:08.913204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1197/2404] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:09.119335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd717221a65f1409d83f10ddfa374ebeb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.119989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd717221a65f1409d83f10ddfa374ebeb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.120605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd717221a65f1409d83f10ddfa374ebeb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.129317 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd717221a65f1409d83f10ddfa374ebeb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.147744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd717221a65f1409d83f10ddfa374ebeb'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.249372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1208/2405] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:08 2022] DELETE /identity/v3/users/d717221a65f1409d83f10ddfa374ebeb => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:09.253905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3485e5f-7455-4b0e-93ad-f2db1ba564be None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.254424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3485e5f-7455-4b0e-93ad-f2db1ba564be None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.254601 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3485e5f-7455-4b0e-93ad-f2db1ba564be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.274287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c3485e5f-7455-4b0e-93ad-f2db1ba564be None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:09.276006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c3485e5f-7455-4b0e-93ad-f2db1ba564be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:09.320735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:09.322902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:09.355145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1198/2406] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:09.365374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3485e5f-7455-4b0e-93ad-f2db1ba564be None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:09.368469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c3485e5f-7455-4b0e-93ad-f2db1ba564be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:09.413547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76888d2e-d5fc-4efe-bc58-a50f4907a52f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:09.419489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:09.420139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.420442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.420699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.422466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:09.432418 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3485e5f-7455-4b0e-93ad-f2db1ba564be None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:09.434430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:09.436096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:09.436224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2433bd6-8643-4585-bca4-65219c49658f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:09.436789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2433bd6-8643-4585-bca4-65219c49658f service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.437074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2433bd6-8643-4585-bca4-65219c49658f service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.437351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2433bd6-8643-4585-bca4-65219c49658f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.448364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1209/2407] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:09.454836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2433bd6-8643-4585-bca4-65219c49658f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:09.454836 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2433bd6-8643-4585-bca4-65219c49658f service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:09.461854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1199/2408] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:09.484244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:09.484244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:09.497140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57f6e2-0326-4ba2-93a3-b37c62839550 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:09.502064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:09.502945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.503232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.503546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] PATH_INFO: `/v3/projects/09ca5219beb44efea7223c9447bfe55c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.504846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ca5219beb44efea7223c9447bfe55c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:09.506766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:09.545428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:09.653748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ca5219beb44efea7223c9447bfe55c', 'request_id': 'req-0a919164-9528-4506-83f8-456ddacdec75'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.660499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ca5219beb44efea7223c9447bfe55c', 'request_id': 'req-0a919164-9528-4506-83f8-456ddacdec75'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.726096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ca5219beb44efea7223c9447bfe55c', 'request_id': 'req-0a919164-9528-4506-83f8-456ddacdec75'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.738087 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] The token cache is being invalidate because project 09ca5219beb44efea7223c9447bfe55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:09.738454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.739197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:09.740968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1210/2409] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:09 2022] DELETE /identity/v3/projects/09ca5219beb44efea7223c9447bfe55c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:09.745434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2433bd6-8643-4585-bca4-65219c49658f service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:09.761241 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2433bd6-8643-4585-bca4-65219c49658f service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:09.817907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:09.818726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.819010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.819308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] PATH_INFO: `/v3/users/1309b6e99eb04aa682fdf0dc2f839c00` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.821115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1309b6e99eb04aa682fdf0dc2f839c00)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:09.823382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:09.895738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:09.897814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:09.913013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a919164-9528-4506-83f8-456ddacdec75 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:09.922141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:09.922141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:09.922141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:09.922141 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] PATH_INFO: `/v3/projects/9a8692bb55544ced870a7b4f1f2c4dcf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:09.923102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a8692bb55544ced870a7b4f1f2c4dcf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:09.924782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.017259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:10.019225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1309b6e99eb04aa682fdf0dc2f839c00'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.019948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1309b6e99eb04aa682fdf0dc2f839c00'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.020560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1309b6e99eb04aa682fdf0dc2f839c00'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.027867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1309b6e99eb04aa682fdf0dc2f839c00'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.040336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-62bdde32-8aa3-4405-9704-e7a3095af6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1309b6e99eb04aa682fdf0dc2f839c00'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.079313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a8692bb55544ced870a7b4f1f2c4dcf', 'request_id': 'req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.084846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a8692bb55544ced870a7b4f1f2c4dcf', 'request_id': 'req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.147511 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1200/2410] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:09 2022] DELETE /identity/v3/users/1309b6e99eb04aa682fdf0dc2f839c00 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:10.198220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a8692bb55544ced870a7b4f1f2c4dcf', 'request_id': 'req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.211788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] The token cache is being invalidate because project 9a8692bb55544ced870a7b4f1f2c4dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:10.212216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.212979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b5618dc4-745f-445d-ba5e-c3cf2ea1cdf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.218690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1211/2411] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:09 2022] DELETE /identity/v3/projects/9a8692bb55544ced870a7b4f1f2c4dcf => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:10.220828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19557b5-4575-4c64-a6f2-fc4890d7ed9f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.221113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19557b5-4575-4c64-a6f2-fc4890d7ed9f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.221504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19557b5-4575-4c64-a6f2-fc4890d7ed9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.242307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c19557b5-4575-4c64-a6f2-fc4890d7ed9f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:10.243792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c19557b5-4575-4c64-a6f2-fc4890d7ed9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:10.308175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1201/2412] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:10.317336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2569049-24fe-4815-a33a-e1a5f75e846a None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.317634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2569049-24fe-4815-a33a-e1a5f75e846a None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.317914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2569049-24fe-4815-a33a-e1a5f75e846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.337996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d2569049-24fe-4815-a33a-e1a5f75e846a None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:10.339171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d2569049-24fe-4815-a33a-e1a5f75e846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:10.347212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1212/2413] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:10.351592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19557b5-4575-4c64-a6f2-fc4890d7ed9f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.361656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19557b5-4575-4c64-a6f2-fc4890d7ed9f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.367832 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.368890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.369170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.369443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.370339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:10.370476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.372106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.386727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1202/2414] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:59:10.394955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2569049-24fe-4815-a33a-e1a5f75e846a None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.408611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2569049-24fe-4815-a33a-e1a5f75e846a None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.414571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.414571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.414571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.414968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.415486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.417368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.538320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1213/2415] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:10 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:10.542294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.552480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2677a39-a1b1-4dee-a4a8-71114473af63 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.567191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.568970 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:10.616687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3000d-b63d-4534-b334-be92ef32fc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.617931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3000d-b63d-4534-b334-be92ef32fc22 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.618232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3000d-b63d-4534-b334-be92ef32fc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.618507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3000d-b63d-4534-b334-be92ef32fc22 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.619206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3000d-b63d-4534-b334-be92ef32fc22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.621145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3000d-b63d-4534-b334-be92ef32fc22 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.639118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e56580-0996-447d-a6f6-974e1bb29ce6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.643777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.644780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.645069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.645344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] PATH_INFO: `/v3/projects/794ddf8b808747d79588dede429b007d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.646520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=794ddf8b808747d79588dede429b007d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.648444 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.656254 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:10.736835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1203/2416] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:10 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:10.740686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3000d-b63d-4534-b334-be92ef32fc22 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.750253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3000d-b63d-4534-b334-be92ef32fc22 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.754972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc73b4e-94ed-4568-9997-13a0961f8a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.755643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc73b4e-94ed-4568-9997-13a0961f8a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.755932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc73b4e-94ed-4568-9997-13a0961f8a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.756214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc73b4e-94ed-4568-9997-13a0961f8a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.756939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc73b4e-94ed-4568-9997-13a0961f8a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.758628 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc73b4e-94ed-4568-9997-13a0961f8a87 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.771620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1204/2417] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:10.772982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc73b4e-94ed-4568-9997-13a0961f8a87 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.777658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794ddf8b808747d79588dede429b007d', 'request_id': 'req-fddc458e-dcbc-4662-99f6-909953e77012'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.783037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794ddf8b808747d79588dede429b007d', 'request_id': 'req-fddc458e-dcbc-4662-99f6-909953e77012'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.783175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc73b4e-94ed-4568-9997-13a0961f8a87 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.807868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:10.839452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf02541-9169-4a35-a0d4-11b581657967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.840112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf02541-9169-4a35-a0d4-11b581657967 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.840396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf02541-9169-4a35-a0d4-11b581657967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.840673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf02541-9169-4a35-a0d4-11b581657967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.841391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf02541-9169-4a35-a0d4-11b581657967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.843051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf02541-9169-4a35-a0d4-11b581657967 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.849740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794ddf8b808747d79588dede429b007d', 'request_id': 'req-fddc458e-dcbc-4662-99f6-909953e77012'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.852903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1205/2418] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:10.857005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf02541-9169-4a35-a0d4-11b581657967 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.861861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] The token cache is being invalidate because project 794ddf8b808747d79588dede429b007d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:10.862353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.863085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-fddc458e-dcbc-4662-99f6-909953e77012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:10.868336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf02541-9169-4a35-a0d4-11b581657967 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.868499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1214/2419] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:10 2022] DELETE /identity/v3/projects/794ddf8b808747d79588dede429b007d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:10.869910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e3eb0-4ffc-4821-979c-3b8e293d4710 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.870370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e3eb0-4ffc-4821-979c-3b8e293d4710 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.870823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e3eb0-4ffc-4821-979c-3b8e293d4710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.873933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311b6a4-5c04-4e44-835a-db3289c64947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.874878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311b6a4-5c04-4e44-835a-db3289c64947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.875188 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311b6a4-5c04-4e44-835a-db3289c64947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.875480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311b6a4-5c04-4e44-835a-db3289c64947 admin admin] PATH_INFO: `/v3/projects/791d2a150ef44ce39f4175f188dc9e21/users/6b08f44b2dfb4b91996a89d4ed57a1d3/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.890898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-138e3eb0-4ffc-4821-979c-3b8e293d4710 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:10.892167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-138e3eb0-4ffc-4821-979c-3b8e293d4710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:10.895777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311b6a4-5c04-4e44-835a-db3289c64947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=791d2a150ef44ce39f4175f188dc9e21, user_id=6b08f44b2dfb4b91996a89d4ed57a1d3, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.899239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311b6a4-5c04-4e44-835a-db3289c64947 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.900768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1215/2420] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:10.909753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc6332-7363-48af-a95c-78b2b24a12f0 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.910049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc6332-7363-48af-a95c-78b2b24a12f0 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.910335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc6332-7363-48af-a95c-78b2b24a12f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.927862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-50dc6332-7363-48af-a95c-78b2b24a12f0 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:10.929062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-50dc6332-7363-48af-a95c-78b2b24a12f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:10.936976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1216/2421] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:10.940886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc6332-7363-48af-a95c-78b2b24a12f0 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.950475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc6332-7363-48af-a95c-78b2b24a12f0 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:10.955112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0feff-daa7-4da8-990a-0251754d3f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:10.955764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0feff-daa7-4da8-990a-0251754d3f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.956050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0feff-daa7-4da8-990a-0251754d3f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.956437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0feff-daa7-4da8-990a-0251754d3f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:10.957157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0feff-daa7-4da8-990a-0251754d3f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:10.957469 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0feff-daa7-4da8-990a-0251754d3f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:10.959052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0feff-daa7-4da8-990a-0251754d3f2b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:10.969841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1206/2422] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:10 2022] PUT /identity/v3/projects/791d2a150ef44ce39f4175f188dc9e21/users/6b08f44b2dfb4b91996a89d4ed57a1d3/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:10.971505 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1217/2423] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:59:10.979579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311b6a4-5c04-4e44-835a-db3289c64947 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:10.980055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca7da9-028e-414d-a61c-d5fc5f6bece9 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:10.980349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca7da9-028e-414d-a61c-d5fc5f6bece9 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:10.980672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca7da9-028e-414d-a61c-d5fc5f6bece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.006126 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f1ca7da9-028e-414d-a61c-d5fc5f6bece9 None None] MFA Rules not processed for user `6b08f44b2dfb4b91996a89d4ed57a1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:11.008824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f1ca7da9-028e-414d-a61c-d5fc5f6bece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:11.041987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311b6a4-5c04-4e44-835a-db3289c64947 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.046727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde4601-8caa-413d-bbe6-b6ac2e09e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.047410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde4601-8caa-413d-bbe6-b6ac2e09e1ef admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.047698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde4601-8caa-413d-bbe6-b6ac2e09e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.047977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde4601-8caa-413d-bbe6-b6ac2e09e1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.048822 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde4601-8caa-413d-bbe6-b6ac2e09e1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.050712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde4601-8caa-413d-bbe6-b6ac2e09e1ef admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.075028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1218/2424] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:11.079211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca7da9-028e-414d-a61c-d5fc5f6bece9 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.080528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f1ca7da9-028e-414d-a61c-d5fc5f6bece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:11.094235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca7da9-028e-414d-a61c-d5fc5f6bece9 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.098921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c5f57-ccf9-4009-818b-242b6d57d006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.099593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c5f57-ccf9-4009-818b-242b6d57d006 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.099930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c5f57-ccf9-4009-818b-242b6d57d006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.100160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c5f57-ccf9-4009-818b-242b6d57d006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.100844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c5f57-ccf9-4009-818b-242b6d57d006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.102862 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c5f57-ccf9-4009-818b-242b6d57d006 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.155740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1207/2425] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:10 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:11.159627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde4601-8caa-413d-bbe6-b6ac2e09e1ef admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.187930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1219/2426] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:11 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:11.189245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c5f57-ccf9-4009-818b-242b6d57d006 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.232080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde4601-8caa-413d-bbe6-b6ac2e09e1ef admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.232678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:11.253095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c5f57-ccf9-4009-818b-242b6d57d006 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.258394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae621964-bd62-4f78-9843-fd1051e2c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.259067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae621964-bd62-4f78-9843-fd1051e2c672 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.259430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae621964-bd62-4f78-9843-fd1051e2c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.259725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae621964-bd62-4f78-9843-fd1051e2c672 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.260400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae621964-bd62-4f78-9843-fd1051e2c672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.261787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:11.262497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae621964-bd62-4f78-9843-fd1051e2c672 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.293901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cae2fe-aa1c-483a-b8db-67ad429564b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.294606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cae2fe-aa1c-483a-b8db-67ad429564b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.294899 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cae2fe-aa1c-483a-b8db-67ad429564b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.295401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cae2fe-aa1c-483a-b8db-67ad429564b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.296258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cae2fe-aa1c-483a-b8db-67ad429564b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.298624 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cae2fe-aa1c-483a-b8db-67ad429564b8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.376779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1220/2427] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:11 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:11.380301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae621964-bd62-4f78-9843-fd1051e2c672 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.387867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1208/2428] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:11 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:11.391354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae621964-bd62-4f78-9843-fd1051e2c672 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.392015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cae2fe-aa1c-483a-b8db-67ad429564b8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.396122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefadec3-6a34-496a-ab2e-565790caa48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.396787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefadec3-6a34-496a-ab2e-565790caa48d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.397074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefadec3-6a34-496a-ab2e-565790caa48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.397350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefadec3-6a34-496a-ab2e-565790caa48d admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.398085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefadec3-6a34-496a-ab2e-565790caa48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.399787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefadec3-6a34-496a-ab2e-565790caa48d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.402039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cae2fe-aa1c-483a-b8db-67ad429564b8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.406772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ab23-cb41-443f-b23e-bb7dc5641c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.408557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ab23-cb41-443f-b23e-bb7dc5641c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.408557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ab23-cb41-443f-b23e-bb7dc5641c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.408557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ab23-cb41-443f-b23e-bb7dc5641c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.408887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ab23-cb41-443f-b23e-bb7dc5641c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.413205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1221/2429] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:11.413371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ab23-cb41-443f-b23e-bb7dc5641c29 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.414238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefadec3-6a34-496a-ab2e-565790caa48d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.423372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1209/2430] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:11.423787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefadec3-6a34-496a-ab2e-565790caa48d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.424197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ab23-cb41-443f-b23e-bb7dc5641c29 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.434342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df79f66-b7fb-4870-9d25-5d75625c8d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.435801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ab23-cb41-443f-b23e-bb7dc5641c29 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.435923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df79f66-b7fb-4870-9d25-5d75625c8d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.435923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df79f66-b7fb-4870-9d25-5d75625c8d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.435923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df79f66-b7fb-4870-9d25-5d75625c8d0d admin admin] PATH_INFO: `/v3/projects/72aad16e04ee4334b2292fe8061bd6a7/users/40510ce875ad4917bc23c97794443c76/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.439217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363b24-0a4d-423f-bfe4-8d6c1b964acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.439903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363b24-0a4d-423f-bfe4-8d6c1b964acb admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.440214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363b24-0a4d-423f-bfe4-8d6c1b964acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.440474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54363b24-0a4d-423f-bfe4-8d6c1b964acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.441235 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54363b24-0a4d-423f-bfe4-8d6c1b964acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.442837 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54363b24-0a4d-423f-bfe4-8d6c1b964acb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.451777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df79f66-b7fb-4870-9d25-5d75625c8d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72aad16e04ee4334b2292fe8061bd6a7, user_id=40510ce875ad4917bc23c97794443c76, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.457205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1210/2431] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:11.457388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df79f66-b7fb-4870-9d25-5d75625c8d0d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.458167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363b24-0a4d-423f-bfe4-8d6c1b964acb admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.467812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54363b24-0a4d-423f-bfe4-8d6c1b964acb admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.472523 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4099ab0-4479-4887-af49-ddf155af2001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.473408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4099ab0-4479-4887-af49-ddf155af2001 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.473693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4099ab0-4479-4887-af49-ddf155af2001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.473972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4099ab0-4479-4887-af49-ddf155af2001 admin admin] PATH_INFO: `/v3/projects/f91ddb2bb43747569b5ca0ac70fed6b3/users/1fa500524b754ce7895f2bd30d3fbf2d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.489279 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4099ab0-4479-4887-af49-ddf155af2001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f91ddb2bb43747569b5ca0ac70fed6b3, user_id=1fa500524b754ce7895f2bd30d3fbf2d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.491972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4099ab0-4479-4887-af49-ddf155af2001 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.569652 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1222/2432] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:11 2022] PUT /identity/v3/projects/72aad16e04ee4334b2292fe8061bd6a7/users/40510ce875ad4917bc23c97794443c76/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:11.573816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df79f66-b7fb-4870-9d25-5d75625c8d0d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.580599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1211/2433] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:11 2022] PUT /identity/v3/projects/f91ddb2bb43747569b5ca0ac70fed6b3/users/1fa500524b754ce7895f2bd30d3fbf2d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:11.590594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd241-8899-4d19-a093-61edc84d898f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.590852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd241-8899-4d19-a093-61edc84d898f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.591136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd241-8899-4d19-a093-61edc84d898f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.618172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4cedd241-8899-4d19-a093-61edc84d898f None None] MFA Rules not processed for user `1fa500524b754ce7895f2bd30d3fbf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:11.619420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4cedd241-8899-4d19-a093-61edc84d898f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:11.633683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df79f66-b7fb-4870-9d25-5d75625c8d0d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.682799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1212/2434] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:11.687203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedd241-8899-4d19-a093-61edc84d898f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.689022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f958b-6da3-4feb-8055-7f64bf156343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.689723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f958b-6da3-4feb-8055-7f64bf156343 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.690007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f958b-6da3-4feb-8055-7f64bf156343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.690284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f958b-6da3-4feb-8055-7f64bf156343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.691005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f958b-6da3-4feb-8055-7f64bf156343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.692902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f958b-6da3-4feb-8055-7f64bf156343 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.697819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedd241-8899-4d19-a093-61edc84d898f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.702422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb70a0-0cb0-4c4f-aa0f-af1e18e31c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.703123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb70a0-0cb0-4c4f-aa0f-af1e18e31c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.706310 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1223/2435] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:11.706456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb70a0-0cb0-4c4f-aa0f-af1e18e31c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.706456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb70a0-0cb0-4c4f-aa0f-af1e18e31c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.706456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb70a0-0cb0-4c4f-aa0f-af1e18e31c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.706456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb70a0-0cb0-4c4f-aa0f-af1e18e31c18 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.707326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f958b-6da3-4feb-8055-7f64bf156343 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.717693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f958b-6da3-4feb-8055-7f64bf156343 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.722320 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d25016-9a0d-45fe-b622-e095f995ccc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.723206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d25016-9a0d-45fe-b622-e095f995ccc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.723524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d25016-9a0d-45fe-b622-e095f995ccc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.723802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d25016-9a0d-45fe-b622-e095f995ccc0 admin admin] PATH_INFO: `/v3/projects/72aad16e04ee4334b2292fe8061bd6a7/users/40510ce875ad4917bc23c97794443c76/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.725772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d25016-9a0d-45fe-b622-e095f995ccc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72aad16e04ee4334b2292fe8061bd6a7, user_id=40510ce875ad4917bc23c97794443c76, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.728954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d25016-9a0d-45fe-b622-e095f995ccc0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.739017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1224/2436] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:11 2022] PUT /identity/v3/projects/72aad16e04ee4334b2292fe8061bd6a7/users/40510ce875ad4917bc23c97794443c76/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:11.749393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896444-222c-4f40-98e8-17fa10b67e87 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.749688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896444-222c-4f40-98e8-17fa10b67e87 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.749973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896444-222c-4f40-98e8-17fa10b67e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.774899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-44896444-222c-4f40-98e8-17fa10b67e87 None None] MFA Rules not processed for user `40510ce875ad4917bc23c97794443c76`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:11.776168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-44896444-222c-4f40-98e8-17fa10b67e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:11.785010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:11.798069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1213/2437] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:11 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:11.799344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb70a0-0cb0-4c4f-aa0f-af1e18e31c18 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.843715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1225/2438] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:11.844219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:11.857153 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44896444-222c-4f40-98e8-17fa10b67e87 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.858575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-44896444-222c-4f40-98e8-17fa10b67e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:11.861705 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb70a0-0cb0-4c4f-aa0f-af1e18e31c18 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.866379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acf49c-a5ef-4971-ad7b-e6a28f382030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.867043 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acf49c-a5ef-4971-ad7b-e6a28f382030 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.867991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acf49c-a5ef-4971-ad7b-e6a28f382030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.868268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acf49c-a5ef-4971-ad7b-e6a28f382030 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.868950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8acf49c-a5ef-4971-ad7b-e6a28f382030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.870926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8acf49c-a5ef-4971-ad7b-e6a28f382030 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:11.916468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44896444-222c-4f40-98e8-17fa10b67e87 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.963502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8319be-4e17-456f-afb9-3026ed495f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.964230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8319be-4e17-456f-afb9-3026ed495f5d service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.964510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8319be-4e17-456f-afb9-3026ed495f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.964783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8319be-4e17-456f-afb9-3026ed495f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.970786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1214/2439] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:11 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:11.974702 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acf49c-a5ef-4971-ad7b-e6a28f382030 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:11.985979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acf49c-a5ef-4971-ad7b-e6a28f382030 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:11.990667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37515cc0-ab41-4bb4-b257-6ee4e9f0bdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:11.991344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37515cc0-ab41-4bb4-b257-6ee4e9f0bdbe admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:11.991627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37515cc0-ab41-4bb4-b257-6ee4e9f0bdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:11.991905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37515cc0-ab41-4bb4-b257-6ee4e9f0bdbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:11.992664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37515cc0-ab41-4bb4-b257-6ee4e9f0bdbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:11.994244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37515cc0-ab41-4bb4-b257-6ee4e9f0bdbe admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.004021 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1215/2440] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:12.007797 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37515cc0-ab41-4bb4-b257-6ee4e9f0bdbe admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.017218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37515cc0-ab41-4bb4-b257-6ee4e9f0bdbe admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:12.021732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9917ccc-eb38-4285-ad2d-a45e2467c0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:12.022359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9917ccc-eb38-4285-ad2d-a45e2467c0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.022639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9917ccc-eb38-4285-ad2d-a45e2467c0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.022918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9917ccc-eb38-4285-ad2d-a45e2467c0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.024168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9917ccc-eb38-4285-ad2d-a45e2467c0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.025713 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9917ccc-eb38-4285-ad2d-a45e2467c0f2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.025957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8319be-4e17-456f-afb9-3026ed495f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.027424 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8319be-4e17-456f-afb9-3026ed495f5d service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.036211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1216/2441] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:12.039967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1226/2442] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:12.040835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9917ccc-eb38-4285-ad2d-a45e2467c0f2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.050490 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9917ccc-eb38-4285-ad2d-a45e2467c0f2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:12.055115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d0ec0-361c-4a07-b9f9-cb2a40df1b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:12.056035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d0ec0-361c-4a07-b9f9-cb2a40df1b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.056317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d0ec0-361c-4a07-b9f9-cb2a40df1b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.056590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d0ec0-361c-4a07-b9f9-cb2a40df1b15 admin admin] PATH_INFO: `/v3/projects/2a603f997bb64470b30cb10781a2c83e/users/dbbb0206be0b473083fd20c6383f3af1/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.071840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d0ec0-361c-4a07-b9f9-cb2a40df1b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a603f997bb64470b30cb10781a2c83e, user_id=dbbb0206be0b473083fd20c6383f3af1, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.078239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d0ec0-361c-4a07-b9f9-cb2a40df1b15 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.171498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1217/2443] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:12 2022] PUT /identity/v3/projects/2a603f997bb64470b30cb10781a2c83e/users/dbbb0206be0b473083fd20c6383f3af1/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:12.183063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654cc58-7e63-4b84-808b-4fb784031fdb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.183689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654cc58-7e63-4b84-808b-4fb784031fdb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.183978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654cc58-7e63-4b84-808b-4fb784031fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.209528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:12.213713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-9654cc58-7e63-4b84-808b-4fb784031fdb None None] MFA Rules not processed for user `dbbb0206be0b473083fd20c6383f3af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:12.216472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9654cc58-7e63-4b84-808b-4fb784031fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:12.293251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1227/2444] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:12.297647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d0ec0-361c-4a07-b9f9-cb2a40df1b15 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.367707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d0ec0-361c-4a07-b9f9-cb2a40df1b15 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:12.382006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29277df8-2cad-449f-9184-cf5cf9740ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:12.382006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29277df8-2cad-449f-9184-cf5cf9740ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.382006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29277df8-2cad-449f-9184-cf5cf9740ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.382006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29277df8-2cad-449f-9184-cf5cf9740ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.383407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29277df8-2cad-449f-9184-cf5cf9740ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.383407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29277df8-2cad-449f-9184-cf5cf9740ba6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.461644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1218/2445] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:12 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:12.462875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654cc58-7e63-4b84-808b-4fb784031fdb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.480301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654cc58-7e63-4b84-808b-4fb784031fdb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:12.534764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f37ca-7066-4b12-a5ac-0c911d5ec40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:12.535462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f37ca-7066-4b12-a5ac-0c911d5ec40a admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.535750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f37ca-7066-4b12-a5ac-0c911d5ec40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.536029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f37ca-7066-4b12-a5ac-0c911d5ec40a admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.536765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f37ca-7066-4b12-a5ac-0c911d5ec40a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.538679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f37ca-7066-4b12-a5ac-0c911d5ec40a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.634181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1228/2446] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:12 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:12.638176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29277df8-2cad-449f-9184-cf5cf9740ba6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.649159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29277df8-2cad-449f-9184-cf5cf9740ba6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:12.653743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86156ad8-364f-4b98-84b9-01d694ce5e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:12.654394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86156ad8-364f-4b98-84b9-01d694ce5e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.654741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86156ad8-364f-4b98-84b9-01d694ce5e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.655191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86156ad8-364f-4b98-84b9-01d694ce5e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.655883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86156ad8-364f-4b98-84b9-01d694ce5e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.657503 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86156ad8-364f-4b98-84b9-01d694ce5e90 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.669436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1219/2447] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:12.675676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f37ca-7066-4b12-a5ac-0c911d5ec40a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.685382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f37ca-7066-4b12-a5ac-0c911d5ec40a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:12.690374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3df17-1990-4975-bbf7-6bce1356d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:12.691278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3df17-1990-4975-bbf7-6bce1356d16b admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.691653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3df17-1990-4975-bbf7-6bce1356d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.691929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3df17-1990-4975-bbf7-6bce1356d16b admin admin] PATH_INFO: `/v3/projects/ae4927ee185d46c19eebe496d07f934f/users/f8f887ab2a2047caa830392036472de1/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.707877 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3df17-1990-4975-bbf7-6bce1356d16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4927ee185d46c19eebe496d07f934f, user_id=f8f887ab2a2047caa830392036472de1, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.710614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3df17-1990-4975-bbf7-6bce1356d16b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.803858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1229/2448] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:12 2022] PUT /identity/v3/projects/ae4927ee185d46c19eebe496d07f934f/users/f8f887ab2a2047caa830392036472de1/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:12.805390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86156ad8-364f-4b98-84b9-01d694ce5e90 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.816051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86156ad8-364f-4b98-84b9-01d694ce5e90 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:12.873449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b72441-65f3-4f49-9e21-72406b6f0333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:12.874113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72441-65f3-4f49-9e21-72406b6f0333 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.874399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72441-65f3-4f49-9e21-72406b6f0333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.874679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72441-65f3-4f49-9e21-72406b6f0333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.875434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b72441-65f3-4f49-9e21-72406b6f0333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.877029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b72441-65f3-4f49-9e21-72406b6f0333 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.878333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:12.887983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1220/2449] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:12.892273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3df17-1990-4975-bbf7-6bce1356d16b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:12.905009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3df17-1990-4975-bbf7-6bce1356d16b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:12.910105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39723d98-60d0-48ab-8311-bbe4616217fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:12.911038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39723d98-60d0-48ab-8311-bbe4616217fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.911491 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39723d98-60d0-48ab-8311-bbe4616217fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.911777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39723d98-60d0-48ab-8311-bbe4616217fa admin admin] PATH_INFO: `/v3/projects/ae4927ee185d46c19eebe496d07f934f/users/f8f887ab2a2047caa830392036472de1/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.913821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39723d98-60d0-48ab-8311-bbe4616217fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4927ee185d46c19eebe496d07f934f, user_id=f8f887ab2a2047caa830392036472de1, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:12.916506 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39723d98-60d0-48ab-8311-bbe4616217fa admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:12.927780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1230/2450] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:12 2022] PUT /identity/v3/projects/ae4927ee185d46c19eebe496d07f934f/users/f8f887ab2a2047caa830392036472de1/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:12.939056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca96552-fe7f-4573-a8a9-4fc269d33952 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:12.939445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca96552-fe7f-4573-a8a9-4fc269d33952 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:12.939811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca96552-fe7f-4573-a8a9-4fc269d33952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:12.970436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fca96552-fe7f-4573-a8a9-4fc269d33952 None None] MFA Rules not processed for user `f8f887ab2a2047caa830392036472de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:12.977904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fca96552-fe7f-4573-a8a9-4fc269d33952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:12.978339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39723d98-60d0-48ab-8311-bbe4616217fa admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:13.044567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:13.050312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39723d98-60d0-48ab-8311-bbe4616217fa admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:13.058292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:13.059209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1221/2451] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:13.059426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:13.059776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:13.063820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] PATH_INFO: `/v3/users/6b08f44b2dfb4b91996a89d4ed57a1d3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:13.063820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b08f44b2dfb4b91996a89d4ed57a1d3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:13.064131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:13.071941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca96552-fe7f-4573-a8a9-4fc269d33952 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:13.073373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fca96552-fe7f-4573-a8a9-4fc269d33952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:13.138396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca96552-fe7f-4573-a8a9-4fc269d33952 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:13.142154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cab93c-fde0-422d-b7e1-816a329a7eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:13.142795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab93c-fde0-422d-b7e1-816a329a7eca service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:13.143080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab93c-fde0-422d-b7e1-816a329a7eca service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:13.143395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab93c-fde0-422d-b7e1-816a329a7eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:13.153048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cab93c-fde0-422d-b7e1-816a329a7eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:13.154474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cab93c-fde0-422d-b7e1-816a329a7eca service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:13.164795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1222/2452] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:13.268973 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b08f44b2dfb4b91996a89d4ed57a1d3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:13.269550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b08f44b2dfb4b91996a89d4ed57a1d3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:13.270173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b08f44b2dfb4b91996a89d4ed57a1d3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:13.277526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b08f44b2dfb4b91996a89d4ed57a1d3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:13.290757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f37a9065-dc72-42f2-ba83-de70a52bc815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b08f44b2dfb4b91996a89d4ed57a1d3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:13.345252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cab93c-fde0-422d-b7e1-816a329a7eca service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:13.354461 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cab93c-fde0-422d-b7e1-816a329a7eca service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:13.361064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:13.362368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:13.362368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:13.362368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:13.364182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:13.394432 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1231/2453] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:12 2022] DELETE /identity/v3/users/6b08f44b2dfb4b91996a89d4ed57a1d3 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:13.400959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3ad5-783d-4be5-9866-b56be80611f4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:13.401679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3ad5-783d-4be5-9866-b56be80611f4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:13.401967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3ad5-783d-4be5-9866-b56be80611f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:13.422553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-b03c3ad5-783d-4be5-9866-b56be80611f4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:13.424038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b03c3ad5-783d-4be5-9866-b56be80611f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:13.430134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:13.431414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:13.440486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1223/2454] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:13.488836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1232/2455] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:13.500228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:13.505329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:13.547280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:13.587828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023c3c3-2569-4a6d-aaec-5a0586bd989c service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:13.643304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95ee9c-a5d3-4fb1-b51c-ed026d2c7c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:13.644019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95ee9c-a5d3-4fb1-b51c-ed026d2c7c27 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:13.644305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95ee9c-a5d3-4fb1-b51c-ed026d2c7c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:13.644580 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95ee9c-a5d3-4fb1-b51c-ed026d2c7c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:13.654724 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95ee9c-a5d3-4fb1-b51c-ed026d2c7c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:13.656288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95ee9c-a5d3-4fb1-b51c-ed026d2c7c27 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:13.666693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1224/2456] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:14.089211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c3ad5-783d-4be5-9866-b56be80611f4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:14.089211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b03c3ad5-783d-4be5-9866-b56be80611f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:14.104130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c3ad5-783d-4be5-9866-b56be80611f4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:14.108831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:14.109687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:14.109966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:14.110300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] PATH_INFO: `/v3/projects/791d2a150ef44ce39f4175f188dc9e21` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:14.111589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=791d2a150ef44ce39f4175f188dc9e21)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:14.113511 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:14.251208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '791d2a150ef44ce39f4175f188dc9e21', 'request_id': 'req-93a6489e-072b-4bda-b0c3-7a920a0c6020'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.256959 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '791d2a150ef44ce39f4175f188dc9e21', 'request_id': 'req-93a6489e-072b-4bda-b0c3-7a920a0c6020'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.330667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '791d2a150ef44ce39f4175f188dc9e21', 'request_id': 'req-93a6489e-072b-4bda-b0c3-7a920a0c6020'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.338090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:14.342642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] The token cache is being invalidate because project 791d2a150ef44ce39f4175f188dc9e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:14.343044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.343719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.345842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1233/2457] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:14 2022] DELETE /identity/v3/projects/791d2a150ef44ce39f4175f188dc9e21 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:14.349934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95ee9c-a5d3-4fb1-b51c-ed026d2c7c27 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:14.360066 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95ee9c-a5d3-4fb1-b51c-ed026d2c7c27 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:14.422298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:14.423201 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:14.423514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:14.423796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] PATH_INFO: `/v3/users/40510ce875ad4917bc23c97794443c76` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:14.424995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40510ce875ad4917bc23c97794443c76)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:14.426975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:14.635109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40510ce875ad4917bc23c97794443c76'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.635838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40510ce875ad4917bc23c97794443c76'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.636443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40510ce875ad4917bc23c97794443c76'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.643213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40510ce875ad4917bc23c97794443c76'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.654826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ca92cdfe-556d-427e-871c-05843201d742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40510ce875ad4917bc23c97794443c76'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:14.732229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:14.736485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1225/2458] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:14 2022] DELETE /identity/v3/users/40510ce875ad4917bc23c97794443c76 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:15.138495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:15.138495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:15.152667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6489e-072b-4bda-b0c3-7a920a0c6020 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:15.209093 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:15.209958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.210251 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.210527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] PATH_INFO: `/v3/projects/72aad16e04ee4334b2292fe8061bd6a7` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.212267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72aad16e04ee4334b2292fe8061bd6a7)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:15.214173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:15.311306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72aad16e04ee4334b2292fe8061bd6a7', 'request_id': 'req-1305ed0c-cb90-4dad-9014-302e3be38a09'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:15.316697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72aad16e04ee4334b2292fe8061bd6a7', 'request_id': 'req-1305ed0c-cb90-4dad-9014-302e3be38a09'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:15.393811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72aad16e04ee4334b2292fe8061bd6a7', 'request_id': 'req-1305ed0c-cb90-4dad-9014-302e3be38a09'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:15.406313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] The token cache is being invalidate because project 72aad16e04ee4334b2292fe8061bd6a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:15.406606 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:15.407243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1305ed0c-cb90-4dad-9014-302e3be38a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:15.408825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1234/2459] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:15 2022] DELETE /identity/v3/projects/72aad16e04ee4334b2292fe8061bd6a7 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:15.414648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef7bcb-0a10-46b7-8937-bfb876ef7275 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.415121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef7bcb-0a10-46b7-8937-bfb876ef7275 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.415569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef7bcb-0a10-46b7-8937-bfb876ef7275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.436695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-10ef7bcb-0a10-46b7-8937-bfb876ef7275 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:15.437897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-10ef7bcb-0a10-46b7-8937-bfb876ef7275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:15.496559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1226/2460] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:15.553601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b76b7-3203-47c0-9d23-b4bcd1586cc5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.553918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b76b7-3203-47c0-9d23-b4bcd1586cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.554200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b76b7-3203-47c0-9d23-b4bcd1586cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.573350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-388b76b7-3203-47c0-9d23-b4bcd1586cc5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:15.574586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-388b76b7-3203-47c0-9d23-b4bcd1586cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:15.583147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1235/2461] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:15.587509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef7bcb-0a10-46b7-8937-bfb876ef7275 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:15.597259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef7bcb-0a10-46b7-8937-bfb876ef7275 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:15.602401 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:15.603041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.603355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.603631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.604353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:15.604658 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:15.606234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:15.622617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1227/2462] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:59:15.623981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b76b7-3203-47c0-9d23-b4bcd1586cc5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:15.633444 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b76b7-3203-47c0-9d23-b4bcd1586cc5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:15.638377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab077ab-6d9e-4b99-b1b2-58fc42a8715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:15.639039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab077ab-6d9e-4b99-b1b2-58fc42a8715a admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.639381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab077ab-6d9e-4b99-b1b2-58fc42a8715a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.639666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab077ab-6d9e-4b99-b1b2-58fc42a8715a admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.640368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab077ab-6d9e-4b99-b1b2-58fc42a8715a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:15.642280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab077ab-6d9e-4b99-b1b2-58fc42a8715a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:15.747837 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1236/2463] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:15 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:15.751807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:15.762595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19923fe3-262a-4fa2-911c-8d528cd587a2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:15.820902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14623de-a061-46f0-9fe5-1dba0c7f9831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:15.821829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14623de-a061-46f0-9fe5-1dba0c7f9831 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.822109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14623de-a061-46f0-9fe5-1dba0c7f9831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.822382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14623de-a061-46f0-9fe5-1dba0c7f9831 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.823069 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14623de-a061-46f0-9fe5-1dba0c7f9831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:15.824991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14623de-a061-46f0-9fe5-1dba0c7f9831 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:15.930413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1228/2464] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:15 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:15.934487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab077ab-6d9e-4b99-b1b2-58fc42a8715a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:15.944820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab077ab-6d9e-4b99-b1b2-58fc42a8715a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:15.949797 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1b015-3bb8-4a8a-837a-27d1ac3fd5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:15.950456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1b015-3bb8-4a8a-837a-27d1ac3fd5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.950746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1b015-3bb8-4a8a-837a-27d1ac3fd5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.951028 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1b015-3bb8-4a8a-837a-27d1ac3fd5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.951840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1b015-3bb8-4a8a-837a-27d1ac3fd5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:15.956822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1b015-3bb8-4a8a-837a-27d1ac3fd5a2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:15.964496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1237/2465] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:15.968376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14623de-a061-46f0-9fe5-1dba0c7f9831 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:15.978939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14623de-a061-46f0-9fe5-1dba0c7f9831 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:15.983769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f07090-8e37-48cc-a4e0-cf07692e7502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:15.984432 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07090-8e37-48cc-a4e0-cf07692e7502 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:15.984721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07090-8e37-48cc-a4e0-cf07692e7502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:15.985002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f07090-8e37-48cc-a4e0-cf07692e7502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:15.985733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f07090-8e37-48cc-a4e0-cf07692e7502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:15.987400 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f07090-8e37-48cc-a4e0-cf07692e7502 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.000368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1229/2466] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:16.001489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1b015-3bb8-4a8a-837a-27d1ac3fd5a2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.011488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1b015-3bb8-4a8a-837a-27d1ac3fd5a2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.016056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00617790-2960-4534-81b7-3f538d44f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.016963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00617790-2960-4534-81b7-3f538d44f823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.017246 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00617790-2960-4534-81b7-3f538d44f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.017524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00617790-2960-4534-81b7-3f538d44f823 admin admin] PATH_INFO: `/v3/projects/1dfda7e38cb746fa90fcedd54689f268/users/1e97a1c95f2b4975b706c3f4328478ce/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.032335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00617790-2960-4534-81b7-3f538d44f823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfda7e38cb746fa90fcedd54689f268, user_id=1e97a1c95f2b4975b706c3f4328478ce, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.035014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00617790-2960-4534-81b7-3f538d44f823 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.142545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1238/2467] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:16 2022] PUT /identity/v3/projects/1dfda7e38cb746fa90fcedd54689f268/users/1e97a1c95f2b4975b706c3f4328478ce/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:16.153328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20729637-9936-4be5-9697-960255bbbf75 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.153619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20729637-9936-4be5-9697-960255bbbf75 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.153901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20729637-9936-4be5-9697-960255bbbf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.179729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-20729637-9936-4be5-9697-960255bbbf75 None None] MFA Rules not processed for user `1e97a1c95f2b4975b706c3f4328478ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:16.181020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-20729637-9936-4be5-9697-960255bbbf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:16.234990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:16.243826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1230/2468] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:16.249476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00617790-2960-4534-81b7-3f538d44f823 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.277096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:16.309835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00617790-2960-4534-81b7-3f538d44f823 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.314517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371e845-5c25-4139-8a80-d3b7d57f402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.315174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e845-5c25-4139-8a80-d3b7d57f402f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.315487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e845-5c25-4139-8a80-d3b7d57f402f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.315766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e845-5c25-4139-8a80-d3b7d57f402f admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.316454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371e845-5c25-4139-8a80-d3b7d57f402f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.318358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371e845-5c25-4139-8a80-d3b7d57f402f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.400537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1239/2469] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:16 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:16.404665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20729637-9936-4be5-9697-960255bbbf75 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.415341 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20729637-9936-4be5-9697-960255bbbf75 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.470730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2250358-9373-4d6c-ae23-ec356341dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.471428 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2250358-9373-4d6c-ae23-ec356341dddc admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.471714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2250358-9373-4d6c-ae23-ec356341dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.471991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2250358-9373-4d6c-ae23-ec356341dddc admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.472687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2250358-9373-4d6c-ae23-ec356341dddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.474668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2250358-9373-4d6c-ae23-ec356341dddc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.547642 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1231/2470] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:16 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:16.551643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371e845-5c25-4139-8a80-d3b7d57f402f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.562516 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371e845-5c25-4139-8a80-d3b7d57f402f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.567067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab0423-f38b-47f7-bf37-1c03b3beb0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.567768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab0423-f38b-47f7-bf37-1c03b3beb0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.568047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab0423-f38b-47f7-bf37-1c03b3beb0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.568324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab0423-f38b-47f7-bf37-1c03b3beb0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.569065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab0423-f38b-47f7-bf37-1c03b3beb0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.571669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab0423-f38b-47f7-bf37-1c03b3beb0bd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.582865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1240/2471] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:16.586881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2250358-9373-4d6c-ae23-ec356341dddc admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.596757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2250358-9373-4d6c-ae23-ec356341dddc admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.601363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c7af-83c0-48e0-b5b0-f14f4c489644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.602226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c7af-83c0-48e0-b5b0-f14f4c489644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.602504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c7af-83c0-48e0-b5b0-f14f4c489644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.602771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224c7af-83c0-48e0-b5b0-f14f4c489644 admin admin] PATH_INFO: `/v3/projects/0dd9c94b0ed245cca318781930475c4e/users/e6f06650b9794da6a667f62b6ea13f44/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.617673 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c7af-83c0-48e0-b5b0-f14f4c489644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd9c94b0ed245cca318781930475c4e, user_id=e6f06650b9794da6a667f62b6ea13f44, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.620351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224c7af-83c0-48e0-b5b0-f14f4c489644 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.673544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1232/2472] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:16 2022] PUT /identity/v3/projects/0dd9c94b0ed245cca318781930475c4e/users/e6f06650b9794da6a667f62b6ea13f44/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:16.677493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab0423-f38b-47f7-bf37-1c03b3beb0bd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.688676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab0423-f38b-47f7-bf37-1c03b3beb0bd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.745404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646df4cd-11c7-4ea0-b89a-a7eca2bb74ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.746008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646df4cd-11c7-4ea0-b89a-a7eca2bb74ec admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.746291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646df4cd-11c7-4ea0-b89a-a7eca2bb74ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.746566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646df4cd-11c7-4ea0-b89a-a7eca2bb74ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.747304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646df4cd-11c7-4ea0-b89a-a7eca2bb74ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.748897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646df4cd-11c7-4ea0-b89a-a7eca2bb74ec admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.758853 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1241/2473] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:16.763089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c7af-83c0-48e0-b5b0-f14f4c489644 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.773437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224c7af-83c0-48e0-b5b0-f14f4c489644 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.778111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.779023 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.779336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.779615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] PATH_INFO: `/v3/projects/0dd9c94b0ed245cca318781930475c4e/users/e6f06650b9794da6a667f62b6ea13f44/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.781594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd9c94b0ed245cca318781930475c4e, user_id=e6f06650b9794da6a667f62b6ea13f44, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.784276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.795220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1233/2474] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:16 2022] PUT /identity/v3/projects/0dd9c94b0ed245cca318781930475c4e/users/e6f06650b9794da6a667f62b6ea13f44/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:16.806774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35d333-6ada-4e64-a3b0-7658c9e07156 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.807069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35d333-6ada-4e64-a3b0-7658c9e07156 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.807389 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35d333-6ada-4e64-a3b0-7658c9e07156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.832693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fc35d333-6ada-4e64-a3b0-7658c9e07156 None None] MFA Rules not processed for user `e6f06650b9794da6a667f62b6ea13f44`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:16.833881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fc35d333-6ada-4e64-a3b0-7658c9e07156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:16.902120 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1242/2475] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:16.913328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:16.915609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:16.970616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236676cb-f2cf-4813-a144-c0579d1d88e5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:16.974224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:16.974855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:16.975133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:16.975434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:16.984829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:16.986253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:16.996358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1234/2476] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:17.207544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35d333-6ada-4e64-a3b0-7658c9e07156 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:17.217072 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35d333-6ada-4e64-a3b0-7658c9e07156 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:17.221849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:17.223644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:17.223644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:17.223644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:17.240163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:17.280623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:17.281982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:17.290916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1243/2477] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:17 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:19.915088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:19.917192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:19.969579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:19.983188 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd7a0c-e40d-47f6-b25a-457c7e29952e service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:19.990929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:19.990929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:19.990929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:19.990929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] PATH_INFO: `/v3/users/1fa500524b754ce7895f2bd30d3fbf2d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:19.991436 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa500524b754ce7895f2bd30d3fbf2d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:19.993244 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:20.093934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:20.095397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:20.154704 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69e282-4781-4aba-8b8b-0ead0b109db8 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:20.158610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa73ef-c472-4a10-a849-2fe11a8d8eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:20.162886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa73ef-c472-4a10-a849-2fe11a8d8eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:20.162886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa73ef-c472-4a10-a849-2fe11a8d8eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:20.162886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa73ef-c472-4a10-a849-2fe11a8d8eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:20.162886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-97fa73ef-c472-4a10-a849-2fe11a8d8eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:20.174940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa73ef-c472-4a10-a849-2fe11a8d8eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:20.176469 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa73ef-c472-4a10-a849-2fe11a8d8eb2 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:20.186905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1244/2478] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:20.206114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa500524b754ce7895f2bd30d3fbf2d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:20.206971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa500524b754ce7895f2bd30d3fbf2d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:20.207808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa500524b754ce7895f2bd30d3fbf2d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:20.216028 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa500524b754ce7895f2bd30d3fbf2d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:20.229150 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa500524b754ce7895f2bd30d3fbf2d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:20.347080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1235/2479] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:19 2022] DELETE /identity/v3/users/1fa500524b754ce7895f2bd30d3fbf2d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:20.348515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd384-b981-400a-89eb-2f1f86fc72cb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:20.348814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd384-b981-400a-89eb-2f1f86fc72cb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:20.349098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd384-b981-400a-89eb-2f1f86fc72cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:20.369363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fa7bd384-b981-400a-89eb-2f1f86fc72cb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:20.369778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fa7bd384-b981-400a-89eb-2f1f86fc72cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:20.437892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1245/2480] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:20.448155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:20.449851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:20.520181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe82450-6372-44d3-b03f-526b60f123fd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:20.523839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccf743-ae75-4e6c-a21f-e56ff5464a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:20.524484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf743-ae75-4e6c-a21f-e56ff5464a00 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:20.524891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf743-ae75-4e6c-a21f-e56ff5464a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:20.525182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccf743-ae75-4e6c-a21f-e56ff5464a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:20.534317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccf743-ae75-4e6c-a21f-e56ff5464a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:20.535803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccf743-ae75-4e6c-a21f-e56ff5464a00 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:20.545784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1236/2481] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:21.123163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd384-b981-400a-89eb-2f1f86fc72cb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:21.124582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fa7bd384-b981-400a-89eb-2f1f86fc72cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:21.137077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd384-b981-400a-89eb-2f1f86fc72cb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:21.141683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:21.147691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:21.147691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:21.147691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] PATH_INFO: `/v3/projects/f91ddb2bb43747569b5ca0ac70fed6b3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:21.147691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f91ddb2bb43747569b5ca0ac70fed6b3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:21.147691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:21.291997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f91ddb2bb43747569b5ca0ac70fed6b3', 'request_id': 'req-a3825793-40b2-477c-a158-59b0d8d878b1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.297404 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:21.298815 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f91ddb2bb43747569b5ca0ac70fed6b3', 'request_id': 'req-a3825793-40b2-477c-a158-59b0d8d878b1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.365783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f91ddb2bb43747569b5ca0ac70fed6b3', 'request_id': 'req-a3825793-40b2-477c-a158-59b0d8d878b1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.390095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] The token cache is being invalidate because project f91ddb2bb43747569b5ca0ac70fed6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:21.390595 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.402465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.407997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1246/2482] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:21 2022] DELETE /identity/v3/projects/f91ddb2bb43747569b5ca0ac70fed6b3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:21.410442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccf743-ae75-4e6c-a21f-e56ff5464a00 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:21.422305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccf743-ae75-4e6c-a21f-e56ff5464a00 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:21.502780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:21.507919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:21.507919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:21.507919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] PATH_INFO: `/v3/users/dbbb0206be0b473083fd20c6383f3af1` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:21.507919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbbb0206be0b473083fd20c6383f3af1)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:21.507919 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:21.523627 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:21.742717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbbb0206be0b473083fd20c6383f3af1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.743467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbbb0206be0b473083fd20c6383f3af1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.744280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbbb0206be0b473083fd20c6383f3af1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.752445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbbb0206be0b473083fd20c6383f3af1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.770954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbbb0206be0b473083fd20c6383f3af1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:21.894858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1237/2483] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:21 2022] DELETE /identity/v3/users/dbbb0206be0b473083fd20c6383f3af1 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:22.335231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:22.335231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:22.346684 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3825793-40b2-477c-a158-59b0d8d878b1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:22.389710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:22.391189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:22.445774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:22.447182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:22.447607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:22.448012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] PATH_INFO: `/v3/users/55243615975f4da385cbd7e1a7fe45cd` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:22.450554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55243615975f4da385cbd7e1a7fe45cd)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:22.452660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:22.506777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496200df-02e8-4173-9710-3ad55dab5f80 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:22.512037 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:22.512922 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:22.513455 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:22.514074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] PATH_INFO: `/v3/projects/2a603f997bb64470b30cb10781a2c83e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:22.515548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a603f997bb64470b30cb10781a2c83e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:22.517651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:22.596377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55243615975f4da385cbd7e1a7fe45cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.597048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55243615975f4da385cbd7e1a7fe45cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.597677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55243615975f4da385cbd7e1a7fe45cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.605333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55243615975f4da385cbd7e1a7fe45cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.617242 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4a73e991-e889-4ded-9718-84d84e277d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55243615975f4da385cbd7e1a7fe45cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.646517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a603f997bb64470b30cb10781a2c83e', 'request_id': 'req-ad61be03-f047-4b16-a7ae-3eb339ab94af'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.646517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a603f997bb64470b30cb10781a2c83e', 'request_id': 'req-ad61be03-f047-4b16-a7ae-3eb339ab94af'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.758005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1247/2484] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:22 2022] DELETE /identity/v3/users/55243615975f4da385cbd7e1a7fe45cd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:22.762884 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623fa7-63d5-4f75-a648-0b7853883738 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:22.763184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623fa7-63d5-4f75-a648-0b7853883738 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:22.763693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623fa7-63d5-4f75-a648-0b7853883738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:22.781088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a7623fa7-63d5-4f75-a648-0b7853883738 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:22.782296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a7623fa7-63d5-4f75-a648-0b7853883738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:22.810134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a603f997bb64470b30cb10781a2c83e', 'request_id': 'req-ad61be03-f047-4b16-a7ae-3eb339ab94af'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.825002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] The token cache is being invalidate because project 2a603f997bb64470b30cb10781a2c83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:22.825704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.827124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:22.830030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1238/2485] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:22 2022] DELETE /identity/v3/projects/2a603f997bb64470b30cb10781a2c83e => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:22.834002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:22.836238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:22.879177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1248/2486] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:22.891122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623fa7-63d5-4f75-a648-0b7853883738 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:22.892538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:22.894601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a7623fa7-63d5-4f75-a648-0b7853883738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:22.907306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:22.952467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61be03-f047-4b16-a7ae-3eb339ab94af admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:22.963968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:22.965125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:22.965579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:22.965817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] PATH_INFO: `/v3/users/f8f887ab2a2047caa830392036472de1` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:22.967216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f887ab2a2047caa830392036472de1)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:22.970347 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:23.024782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623fa7-63d5-4f75-a648-0b7853883738 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:23.029351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:23.029955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:23.030249 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:23.030531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:23.042041 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:23.043562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:23.055642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1249/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:23.200795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f887ab2a2047caa830392036472de1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.201465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f887ab2a2047caa830392036472de1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.202046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f887ab2a2047caa830392036472de1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.208923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f887ab2a2047caa830392036472de1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.224903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f887ab2a2047caa830392036472de1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.361559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1239/2488] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:22 2022] DELETE /identity/v3/users/f8f887ab2a2047caa830392036472de1 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:23.469731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:23.472383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:23.498209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cfdcfd-8bce-465d-b582-1bf2de89c18f service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:23.565008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:23.566034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:23.566620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:23.566900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] PATH_INFO: `/v3/projects/fdbcae1524b644659927d3d6bb80a9b0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:23.568228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdbcae1524b644659927d3d6bb80a9b0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:23.570546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:23.717004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdbcae1524b644659927d3d6bb80a9b0', 'request_id': 'req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.724596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdbcae1524b644659927d3d6bb80a9b0', 'request_id': 'req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.827936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdbcae1524b644659927d3d6bb80a9b0', 'request_id': 'req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.846737 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] The token cache is being invalidate because project fdbcae1524b644659927d3d6bb80a9b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:23.847110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.847899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:23.852377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1250/2489] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:23 2022] DELETE /identity/v3/projects/fdbcae1524b644659927d3d6bb80a9b0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:23.856938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:23.856938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:23.873294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01dd9d-6591-4bd0-99ae-6b52d62608f8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:23.935774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:23.937743 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:23.938620 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:23.939531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:23.939812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:23.940087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] PATH_INFO: `/v3/users/1c43a11942724ea6bd886d602b1da9b9` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:23.941493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c43a11942724ea6bd886d602b1da9b9)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:23.943578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:23.958284 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5a144-de65-4a6d-a6a9-c16a287b5dc5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:23.965067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:23.966288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:23.966585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:23.967293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] PATH_INFO: `/v3/projects/ae4927ee185d46c19eebe496d07f934f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:23.969212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae4927ee185d46c19eebe496d07f934f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:23.971550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:24.167833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:24.178814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:24.182367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c43a11942724ea6bd886d602b1da9b9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.183146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c43a11942724ea6bd886d602b1da9b9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.183369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae4927ee185d46c19eebe496d07f934f', 'request_id': 'req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.183955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c43a11942724ea6bd886d602b1da9b9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.190139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae4927ee185d46c19eebe496d07f934f', 'request_id': 'req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.198685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c43a11942724ea6bd886d602b1da9b9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.222694 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d03fa0d5-8a9b-477a-9648-94b3e09d9096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c43a11942724ea6bd886d602b1da9b9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.250430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae4927ee185d46c19eebe496d07f934f', 'request_id': 'req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.269035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] The token cache is being invalidate because project ae4927ee185d46c19eebe496d07f934f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:24.269384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.270118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.271742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1251/2490] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:23 2022] DELETE /identity/v3/projects/ae4927ee185d46c19eebe496d07f934f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:24.277058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:24.278514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:24.302698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1240/2491] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:23 2022] DELETE /identity/v3/users/1c43a11942724ea6bd886d602b1da9b9 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:24.307651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdaf71-21b5-4338-aeea-357d64e2400c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:24.319671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdaf71-21b5-4338-aeea-357d64e2400c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:24.320016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdaf71-21b5-4338-aeea-357d64e2400c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:24.339394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-6fcdaf71-21b5-4338-aeea-357d64e2400c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:24.340594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6fcdaf71-21b5-4338-aeea-357d64e2400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:24.377844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd39b-09df-4e7e-8fc7-25f1b75b3871 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:24.414196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1241/2492] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:24.415064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdaf71-21b5-4338-aeea-357d64e2400c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:24.420941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6fcdaf71-21b5-4338-aeea-357d64e2400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:24.432653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdaf71-21b5-4338-aeea-357d64e2400c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:24.437682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:24.438546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:24.438829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:24.439102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] PATH_INFO: `/v3/users/cbcc15c786fb43669ac34a4897735c1d` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:24.440377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbcc15c786fb43669ac34a4897735c1d)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:24.442352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:24.457156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c4276-e0f6-4e58-8025-656f647e9cb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:24.457823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c4276-e0f6-4e58-8025-656f647e9cb1 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:24.458111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c4276-e0f6-4e58-8025-656f647e9cb1 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:24.458390 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c4276-e0f6-4e58-8025-656f647e9cb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:24.460361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-de8c4276-e0f6-4e58-8025-656f647e9cb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:24.473476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c4276-e0f6-4e58-8025-656f647e9cb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:24.475098 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c4276-e0f6-4e58-8025-656f647e9cb1 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:24.490097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1252/2493] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:24.491731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d7b2-21c1-4cea-be9d-6aa7cc828817 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:24.492021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d7b2-21c1-4cea-be9d-6aa7cc828817 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:24.492357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d7b2-21c1-4cea-be9d-6aa7cc828817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:24.519437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-dd46d7b2-21c1-4cea-be9d-6aa7cc828817 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:24.520902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dd46d7b2-21c1-4cea-be9d-6aa7cc828817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:24.531040 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1253/2494] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:24.535441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d7b2-21c1-4cea-be9d-6aa7cc828817 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:24.553689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd46d7b2-21c1-4cea-be9d-6aa7cc828817 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:24.561055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:24.561758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:24.562033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:24.562314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:24.563053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:24.563435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:24.565039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:24.579207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1254/2495] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:59:24.583907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:24.591120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbcc15c786fb43669ac34a4897735c1d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.592354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbcc15c786fb43669ac34a4897735c1d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.593353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbcc15c786fb43669ac34a4897735c1d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.594557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15af56-aa65-47fd-b947-3391f8548ae2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:24.599434 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:24.600127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:24.600612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:24.600894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:24.601595 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:24.604008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbcc15c786fb43669ac34a4897735c1d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.604196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:24.619935 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbcc15c786fb43669ac34a4897735c1d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:24.696499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:24.703164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1255/2496] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:24 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:24.707030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:24.708636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:24.728654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1242/2497] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:24 2022] DELETE /identity/v3/users/cbcc15c786fb43669ac34a4897735c1d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:24.781316 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:24.783109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:24.812548 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdafb1-2535-4d9d-9d59-40557b4acbb1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:24.828059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:24.860472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6d7bc-0876-4c09-a6f3-8ce05847e2a6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:24.865787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:24.866716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:24.867002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:24.867336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] PATH_INFO: `/v3/projects/12cadb512244446e94b144b03b673ff6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:24.875579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=12cadb512244446e94b144b03b673ff6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:24.875579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:24.895079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc8eee-4acc-4612-b317-61d349e032ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:24.900330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc8eee-4acc-4612-b317-61d349e032ba admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:24.900330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc8eee-4acc-4612-b317-61d349e032ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:24.900330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc8eee-4acc-4612-b317-61d349e032ba admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:24.900330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc8eee-4acc-4612-b317-61d349e032ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:24.901701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc8eee-4acc-4612-b317-61d349e032ba admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:24.996208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12cadb512244446e94b144b03b673ff6', 'request_id': 'req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.001482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12cadb512244446e94b144b03b673ff6', 'request_id': 'req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.012576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1256/2498] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:24 2022] POST /identity/v3/users => generated 291 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:25.014359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc8eee-4acc-4612-b317-61d349e032ba admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.024103 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc8eee-4acc-4612-b317-61d349e032ba admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.085992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12cadb512244446e94b144b03b673ff6', 'request_id': 'req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.093238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436373c8-194e-43f2-96a6-51f5d410758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.093651 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436373c8-194e-43f2-96a6-51f5d410758c admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.093997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436373c8-194e-43f2-96a6-51f5d410758c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.094580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436373c8-194e-43f2-96a6-51f5d410758c admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.103998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436373c8-194e-43f2-96a6-51f5d410758c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.105840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436373c8-194e-43f2-96a6-51f5d410758c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.106741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] The token cache is being invalidate because project 12cadb512244446e94b144b03b673ff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:25.107185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.108639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.114842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1243/2499] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:24 2022] DELETE /identity/v3/projects/12cadb512244446e94b144b03b673ff6 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:25.114842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.115687 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.123360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1257/2500] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:25.125099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fa713-6446-41f3-8a0b-86d97f022646 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.125397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fa713-6446-41f3-8a0b-86d97f022646 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.125863 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fa713-6446-41f3-8a0b-86d97f022646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.134645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c558f6-3ecb-4c85-8e11-29e5b4c62faa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.140452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.153253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.153253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.153253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] PATH_INFO: `/v3/projects/1328a510264f4374baabad7b5243d8c2` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.153253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1328a510264f4374baabad7b5243d8c2)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.153786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-839fa713-6446-41f3-8a0b-86d97f022646 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:25.153786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-839fa713-6446-41f3-8a0b-86d97f022646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.153999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.156605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1258/2501] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:25.157551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fa713-6446-41f3-8a0b-86d97f022646 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.158775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-839fa713-6446-41f3-8a0b-86d97f022646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.172114 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fa713-6446-41f3-8a0b-86d97f022646 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.177020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ef7a1-3f4d-4822-865e-d77d44ac18d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.177692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef7a1-3f4d-4822-865e-d77d44ac18d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.177980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef7a1-3f4d-4822-865e-d77d44ac18d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.178260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ef7a1-3f4d-4822-865e-d77d44ac18d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.179015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ef7a1-3f4d-4822-865e-d77d44ac18d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.180749 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ef7a1-3f4d-4822-865e-d77d44ac18d2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.191173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1259/2502] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:25.193460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d734740-f653-4f07-91d1-a357c931324b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.193460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d734740-f653-4f07-91d1-a357c931324b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.193460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d734740-f653-4f07-91d1-a357c931324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.216691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-0d734740-f653-4f07-91d1-a357c931324b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:25.216691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0d734740-f653-4f07-91d1-a357c931324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.221416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1260/2503] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:25.222179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d734740-f653-4f07-91d1-a357c931324b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.234177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d734740-f653-4f07-91d1-a357c931324b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.238799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:25.247140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1328a510264f4374baabad7b5243d8c2', 'request_id': 'req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.253228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1328a510264f4374baabad7b5243d8c2', 'request_id': 'req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.293437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3d333-8081-4ce3-a7ad-ced9b645cf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.294336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3d333-8081-4ce3-a7ad-ced9b645cf7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.294834 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3d333-8081-4ce3-a7ad-ced9b645cf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.295129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3d333-8081-4ce3-a7ad-ced9b645cf7f admin admin] PATH_INFO: `/v3/projects/3f3e07278ebe462db9e1bbf84aee9bde/users/545c418079c648d282b8bd9953ea3df0/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.301675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1328a510264f4374baabad7b5243d8c2', 'request_id': 'req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.312611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3d333-8081-4ce3-a7ad-ced9b645cf7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3e07278ebe462db9e1bbf84aee9bde, user_id=545c418079c648d282b8bd9953ea3df0, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.314198 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] The token cache is being invalidate because project 1328a510264f4374baabad7b5243d8c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:25.314406 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.315127 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:25.315322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3d333-8081-4ce3-a7ad-ced9b645cf7f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.316943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1244/2504] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:25 2022] DELETE /identity/v3/projects/1328a510264f4374baabad7b5243d8c2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:25.317771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.319042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.332722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f7d2d-ba24-4bc9-a07a-42a78aa727c9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.337992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae1bdb-768a-47b8-b3bf-df559e47c497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.338635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae1bdb-768a-47b8-b3bf-df559e47c497 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.338916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae1bdb-768a-47b8-b3bf-df559e47c497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.339268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae1bdb-768a-47b8-b3bf-df559e47c497 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.340155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae1bdb-768a-47b8-b3bf-df559e47c497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:25.340464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae1bdb-768a-47b8-b3bf-df559e47c497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.342499 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae1bdb-768a-47b8-b3bf-df559e47c497 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.354923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1245/2505] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:59:25.357562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eae04e-2676-4a73-99fc-2e7d58911e38 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.357562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eae04e-2676-4a73-99fc-2e7d58911e38 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.357562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eae04e-2676-4a73-99fc-2e7d58911e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.374576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-81eae04e-2676-4a73-99fc-2e7d58911e38 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:25.375930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-81eae04e-2676-4a73-99fc-2e7d58911e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.398951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1246/2506] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:25.398951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eae04e-2676-4a73-99fc-2e7d58911e38 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.406462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1261/2507] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:25 2022] PUT /identity/v3/projects/3f3e07278ebe462db9e1bbf84aee9bde/users/545c418079c648d282b8bd9953ea3df0/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:25.409498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47979d9e-9407-4914-863b-daf5447c0325 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.409847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47979d9e-9407-4914-863b-daf5447c0325 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.410190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47979d9e-9407-4914-863b-daf5447c0325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.410907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eae04e-2676-4a73-99fc-2e7d58911e38 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.443741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-47979d9e-9407-4914-863b-daf5447c0325 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:25.451308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-47979d9e-9407-4914-863b-daf5447c0325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.467232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:25.499996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.500576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.500861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.501135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.501850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.503793 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.526075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1262/2508] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:25.529616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7752-b44b-497e-86ef-5cee577725d5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.529616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7752-b44b-497e-86ef-5cee577725d5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.529616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d7752-b44b-497e-86ef-5cee577725d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.557756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fd6d7752-b44b-497e-86ef-5cee577725d5 None None] MFA Rules not processed for user `545c418079c648d282b8bd9953ea3df0`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:25.558908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fd6d7752-b44b-497e-86ef-5cee577725d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.633102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1263/2509] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:25.633974 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d7752-b44b-497e-86ef-5cee577725d5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.644002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d7752-b44b-497e-86ef-5cee577725d5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.649021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.658653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.658653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.658653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.658653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:25.658653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.658653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.659158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1247/2510] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/projects => generated 385 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:25.659158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.659158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:25.669124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1264/2511] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:59:25.669124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.677383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee50bd-36ae-40b6-9190-5e9297a2bcc0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.682963 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:25.733206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7e904-9898-4f16-8f55-ce86953a0160 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.737961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2201-1da7-47e9-892e-a4558925a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.738362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cbe03-7c7d-4165-8b35-78151e21522b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.738619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2201-1da7-47e9-892e-a4558925a20f admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.738909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2201-1da7-47e9-892e-a4558925a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.739038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cbe03-7c7d-4165-8b35-78151e21522b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.739188 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d2201-1da7-47e9-892e-a4558925a20f admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.739348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cbe03-7c7d-4165-8b35-78151e21522b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.739632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cbe03-7c7d-4165-8b35-78151e21522b admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.739906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d2201-1da7-47e9-892e-a4558925a20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.740371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cbe03-7c7d-4165-8b35-78151e21522b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.741866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d2201-1da7-47e9-892e-a4558925a20f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.742274 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cbe03-7c7d-4165-8b35-78151e21522b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.865761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1265/2512] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:25.865966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1248/2513] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:25.866478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cbe03-7c7d-4165-8b35-78151e21522b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.866961 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2201-1da7-47e9-892e-a4558925a20f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.875847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cbe03-7c7d-4165-8b35-78151e21522b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.933355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d2201-1da7-47e9-892e-a4558925a20f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.939113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb4004-6614-4797-b323-a1e216376fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.939825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb4004-6614-4797-b323-a1e216376fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.940107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb4004-6614-4797-b323-a1e216376fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.940380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb4004-6614-4797-b323-a1e216376fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.941112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb4004-6614-4797-b323-a1e216376fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.942732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb4004-6614-4797-b323-a1e216376fe0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.947282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e63f2-f6c1-4a72-bc1c-326af3b792b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.948144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e63f2-f6c1-4a72-bc1c-326af3b792b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.948492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e63f2-f6c1-4a72-bc1c-326af3b792b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.948765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e63f2-f6c1-4a72-bc1c-326af3b792b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.949459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e63f2-f6c1-4a72-bc1c-326af3b792b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.951369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e63f2-f6c1-4a72-bc1c-326af3b792b7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:25.953959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1249/2514] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:25.954865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb4004-6614-4797-b323-a1e216376fe0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:25.972006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb4004-6614-4797-b323-a1e216376fe0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:25.976701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9509-ef0d-41de-8b3c-71df330a607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:25.977363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9509-ef0d-41de-8b3c-71df330a607a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:25.977646 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9509-ef0d-41de-8b3c-71df330a607a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:25.978471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9509-ef0d-41de-8b3c-71df330a607a admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:25.979152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9509-ef0d-41de-8b3c-71df330a607a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:25.981098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9509-ef0d-41de-8b3c-71df330a607a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.067488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1266/2515] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:26.067488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e63f2-f6c1-4a72-bc1c-326af3b792b7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.080387 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1250/2516] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:25 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:26.081284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9509-ef0d-41de-8b3c-71df330a607a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.091172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9509-ef0d-41de-8b3c-71df330a607a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.103175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:26.141855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e63f2-f6c1-4a72-bc1c-326af3b792b7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.148494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db9c28-ac0b-4dd6-b9b8-ea8ac84c29ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.148655 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227ad46-fbe8-4a8e-be29-c09d55e149e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.149298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227ad46-fbe8-4a8e-be29-c09d55e149e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.150717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9c28-ac0b-4dd6-b9b8-ea8ac84c29ee admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.151194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227ad46-fbe8-4a8e-be29-c09d55e149e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.151591 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227ad46-fbe8-4a8e-be29-c09d55e149e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.152361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227ad46-fbe8-4a8e-be29-c09d55e149e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.152606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9c28-ac0b-4dd6-b9b8-ea8ac84c29ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.152839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9c28-ac0b-4dd6-b9b8-ea8ac84c29ee admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.153501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db9c28-ac0b-4dd6-b9b8-ea8ac84c29ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.154164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227ad46-fbe8-4a8e-be29-c09d55e149e7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.155558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db9c28-ac0b-4dd6-b9b8-ea8ac84c29ee admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.168313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1267/2517] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:26.168313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227ad46-fbe8-4a8e-be29-c09d55e149e7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.182330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227ad46-fbe8-4a8e-be29-c09d55e149e7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.187526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77e797-53cc-4314-8f1a-eec5538271c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.188203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77e797-53cc-4314-8f1a-eec5538271c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.188494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77e797-53cc-4314-8f1a-eec5538271c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.188786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77e797-53cc-4314-8f1a-eec5538271c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.189502 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77e797-53cc-4314-8f1a-eec5538271c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.191110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77e797-53cc-4314-8f1a-eec5538271c3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.205824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1268/2518] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:26.206646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77e797-53cc-4314-8f1a-eec5538271c3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.216221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77e797-53cc-4314-8f1a-eec5538271c3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.224205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55907988-18e0-44ae-8e1e-98727ad3916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.225085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55907988-18e0-44ae-8e1e-98727ad3916e admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.231438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55907988-18e0-44ae-8e1e-98727ad3916e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.231438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55907988-18e0-44ae-8e1e-98727ad3916e admin admin] PATH_INFO: `/v3/projects/17c996d4dd874a3cb918eb7fa229a75f/users/6d1d521bb1734f69978297470c4c1f40/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.231438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:26.246300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55907988-18e0-44ae-8e1e-98727ad3916e admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c996d4dd874a3cb918eb7fa229a75f, user_id=6d1d521bb1734f69978297470c4c1f40, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.249174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55907988-18e0-44ae-8e1e-98727ad3916e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.285315 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1251/2519] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:26 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:26.286140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db9c28-ac0b-4dd6-b9b8-ea8ac84c29ee admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.296278 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db9c28-ac0b-4dd6-b9b8-ea8ac84c29ee admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.302074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f034b-5401-4bbc-8dcf-f5cbb6e89da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.302074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f034b-5401-4bbc-8dcf-f5cbb6e89da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.302383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f034b-5401-4bbc-8dcf-f5cbb6e89da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.302502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f034b-5401-4bbc-8dcf-f5cbb6e89da0 admin admin] PATH_INFO: `/v3/projects/85352b6c150b4d0f86d76bca65de5148/users/78b7885809264084bdf438f27c79665e/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.319512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f034b-5401-4bbc-8dcf-f5cbb6e89da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85352b6c150b4d0f86d76bca65de5148, user_id=78b7885809264084bdf438f27c79665e, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.324410 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f034b-5401-4bbc-8dcf-f5cbb6e89da0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.325033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1269/2520] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:26 2022] PUT /identity/v3/projects/17c996d4dd874a3cb918eb7fa229a75f/users/6d1d521bb1734f69978297470c4c1f40/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:26.327824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55907988-18e0-44ae-8e1e-98727ad3916e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.387541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1252/2521] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:26 2022] PUT /identity/v3/projects/85352b6c150b4d0f86d76bca65de5148/users/78b7885809264084bdf438f27c79665e/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:26.388917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ed234-f594-40f7-b8a1-e60cac8ce323 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.389205 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ed234-f594-40f7-b8a1-e60cac8ce323 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.389484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ed234-f594-40f7-b8a1-e60cac8ce323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.403663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55907988-18e0-44ae-8e1e-98727ad3916e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.420678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-108ed234-f594-40f7-b8a1-e60cac8ce323 None None] MFA Rules not processed for user `6d1d521bb1734f69978297470c4c1f40`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:26.422057 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-108ed234-f594-40f7-b8a1-e60cac8ce323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:26.465504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd596a-1972-4249-b58e-97148a95a440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.466185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd596a-1972-4249-b58e-97148a95a440 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.466700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd596a-1972-4249-b58e-97148a95a440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.467542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd596a-1972-4249-b58e-97148a95a440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.467641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd596a-1972-4249-b58e-97148a95a440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.469279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd596a-1972-4249-b58e-97148a95a440 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.483050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1270/2522] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:26.483050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd596a-1972-4249-b58e-97148a95a440 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.495542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd596a-1972-4249-b58e-97148a95a440 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.503136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1253/2523] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:26.503136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ed234-f594-40f7-b8a1-e60cac8ce323 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.505244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af7985-62ba-434c-95d5-937dffcaf143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.505921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af7985-62ba-434c-95d5-937dffcaf143 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.506203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af7985-62ba-434c-95d5-937dffcaf143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.506477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af7985-62ba-434c-95d5-937dffcaf143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.507360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af7985-62ba-434c-95d5-937dffcaf143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.509528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af7985-62ba-434c-95d5-937dffcaf143 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.515376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ed234-f594-40f7-b8a1-e60cac8ce323 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.520439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1271/2524] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:26.521127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af7985-62ba-434c-95d5-937dffcaf143 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.524218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f192c-5b67-464b-bf37-4d08475b61a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.525788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f192c-5b67-464b-bf37-4d08475b61a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.526124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f192c-5b67-464b-bf37-4d08475b61a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.526354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f192c-5b67-464b-bf37-4d08475b61a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.527129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f192c-5b67-464b-bf37-4d08475b61a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.528762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f192c-5b67-464b-bf37-4d08475b61a8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.532260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af7985-62ba-434c-95d5-937dffcaf143 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.537029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274219f7-e0f4-4825-a335-47fcfd694a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.537683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274219f7-e0f4-4825-a335-47fcfd694a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.537967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274219f7-e0f4-4825-a335-47fcfd694a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.538245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274219f7-e0f4-4825-a335-47fcfd694a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.538947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274219f7-e0f4-4825-a335-47fcfd694a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.542863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1254/2525] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:26.542863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f192c-5b67-464b-bf37-4d08475b61a8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.543132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274219f7-e0f4-4825-a335-47fcfd694a5f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.554239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f192c-5b67-464b-bf37-4d08475b61a8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.559329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6dadb-4e2b-41e9-b082-678fed5e5ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.560317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6dadb-4e2b-41e9-b082-678fed5e5ad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.560667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6dadb-4e2b-41e9-b082-678fed5e5ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.560977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6dadb-4e2b-41e9-b082-678fed5e5ad6 admin admin] PATH_INFO: `/v3/projects/85352b6c150b4d0f86d76bca65de5148/users/78b7885809264084bdf438f27c79665e/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.562970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6dadb-4e2b-41e9-b082-678fed5e5ad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85352b6c150b4d0f86d76bca65de5148, user_id=78b7885809264084bdf438f27c79665e, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.565773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6dadb-4e2b-41e9-b082-678fed5e5ad6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.580518 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1255/2526] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:26 2022] PUT /identity/v3/projects/85352b6c150b4d0f86d76bca65de5148/users/78b7885809264084bdf438f27c79665e/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:26.581312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6dadb-4e2b-41e9-b082-678fed5e5ad6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.633722 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:26.637546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1272/2527] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:26 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:26.638851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac4a2-6fb0-4671-9f1c-53dbb152612d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.639144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac4a2-6fb0-4671-9f1c-53dbb152612d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.639453 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac4a2-6fb0-4671-9f1c-53dbb152612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.655604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6dadb-4e2b-41e9-b082-678fed5e5ad6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.667643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a10ac4a2-6fb0-4671-9f1c-53dbb152612d None None] MFA Rules not processed for user `78b7885809264084bdf438f27c79665e`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:26.668879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a10ac4a2-6fb0-4671-9f1c-53dbb152612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:26.716741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:26.726101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.729626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.729626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.729626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] PATH_INFO: `/v3/projects/63924816a0c242ca943117f422a7d778/users/9064cd21f64d4110974d69b4ea94e58f/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.741164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1273/2528] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:26.742200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ac4a2-6fb0-4671-9f1c-53dbb152612d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.742765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=63924816a0c242ca943117f422a7d778, user_id=9064cd21f64d4110974d69b4ea94e58f, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.746107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.753140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ac4a2-6fb0-4671-9f1c-53dbb152612d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.758350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdedf7-f0b3-4b06-9578-caffff6883f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.758554 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdedf7-f0b3-4b06-9578-caffff6883f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.758770 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdedf7-f0b3-4b06-9578-caffff6883f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.759619 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdedf7-f0b3-4b06-9578-caffff6883f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.760303 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fdedf7-f0b3-4b06-9578-caffff6883f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.762195 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fdedf7-f0b3-4b06-9578-caffff6883f9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.803573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1256/2529] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:26 2022] PUT /identity/v3/projects/63924816a0c242ca943117f422a7d778/users/9064cd21f64d4110974d69b4ea94e58f/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:26.804560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.806140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:26.827635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1274/2530] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:26 2022] POST /identity/v3/users => generated 290 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:26.828921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab6a2-e5cd-47cd-9b66-740ff021b2c8 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.829244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab6a2-e5cd-47cd-9b66-740ff021b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.829525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aab6a2-e5cd-47cd-9b66-740ff021b2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.856090 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-92aab6a2-e5cd-47cd-9b66-740ff021b2c8 None None] MFA Rules not processed for user `9064cd21f64d4110974d69b4ea94e58f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:26.857304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-92aab6a2-e5cd-47cd-9b66-740ff021b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:26.869336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd532-376b-471a-92ed-99e37788e02c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:26.873577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:26.874220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:26.876727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:26.876727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:26.923957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1275/2531] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:26.928812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aab6a2-e5cd-47cd-9b66-740ff021b2c8 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.941019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:26.942459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:26.954025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1257/2532] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:26.955108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:26.956483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:26.992768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:27.003893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aab6a2-e5cd-47cd-9b66-740ff021b2c8 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.008989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3827-183c-44d5-9989-3524bb62f599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.009674 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3827-183c-44d5-9989-3524bb62f599 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.014772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3827-183c-44d5-9989-3524bb62f599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.014772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea3827-183c-44d5-9989-3524bb62f599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.014772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3827-183c-44d5-9989-3524bb62f599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.014772 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea3827-183c-44d5-9989-3524bb62f599 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.022917 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1276/2533] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:27.027427 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3827-183c-44d5-9989-3524bb62f599 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.036104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d8bda-a1f6-469f-8800-663266d4a660 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.036784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea3827-183c-44d5-9989-3524bb62f599 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.041350 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc78816-b492-452d-b5ba-e46067d7728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.041747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.042052 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc78816-b492-452d-b5ba-e46067d7728b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.042279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc78816-b492-452d-b5ba-e46067d7728b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.042430 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.042559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc78816-b492-452d-b5ba-e46067d7728b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.042671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.042935 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.043293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc78816-b492-452d-b5ba-e46067d7728b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.044873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc78816-b492-452d-b5ba-e46067d7728b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.050878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.055288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.060870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1277/2534] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:27.062543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1258/2535] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:27.064916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc78816-b492-452d-b5ba-e46067d7728b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.077776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc78816-b492-452d-b5ba-e46067d7728b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.083271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f5954-584a-4bd4-8f76-b97caef11465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.084299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f5954-584a-4bd4-8f76-b97caef11465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.084848 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f5954-584a-4bd4-8f76-b97caef11465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.085216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f5954-584a-4bd4-8f76-b97caef11465 admin admin] PATH_INFO: `/v3/projects/aa90f8f77aa24825b4f1c1b785db3126/users/b4b671df72ac480aa062ea0df9ddda12/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.101039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f5954-584a-4bd4-8f76-b97caef11465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa90f8f77aa24825b4f1c1b785db3126, user_id=b4b671df72ac480aa062ea0df9ddda12, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.104112 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f5954-584a-4bd4-8f76-b97caef11465 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.114896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.116342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:27.130122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053b0f8-d627-4fba-9fa4-b629a8ad9c9b service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.135384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab8404-c7b0-4d8d-b9e8-96dfd8ed8812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.136047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab8404-c7b0-4d8d-b9e8-96dfd8ed8812 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.136335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab8404-c7b0-4d8d-b9e8-96dfd8ed8812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.136708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab8404-c7b0-4d8d-b9e8-96dfd8ed8812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.171510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1278/2536] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:27 2022] PUT /identity/v3/projects/aa90f8f77aa24825b4f1c1b785db3126/users/b4b671df72ac480aa062ea0df9ddda12/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:27.182722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def907c7-9e61-44ec-bd3f-8c8ac014f901 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.183541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def907c7-9e61-44ec-bd3f-8c8ac014f901 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.183830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def907c7-9e61-44ec-bd3f-8c8ac014f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.202706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab8404-c7b0-4d8d-b9e8-96dfd8ed8812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.204350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab8404-c7b0-4d8d-b9e8-96dfd8ed8812 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.212851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:27.216037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-def907c7-9e61-44ec-bd3f-8c8ac014f901 None None] MFA Rules not processed for user `b4b671df72ac480aa062ea0df9ddda12`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:27.217304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-def907c7-9e61-44ec-bd3f-8c8ac014f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:27.219810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1259/2537] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:27.289239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1279/2538] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:27.294424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab8404-c7b0-4d8d-b9e8-96dfd8ed8812 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.305475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab8404-c7b0-4d8d-b9e8-96dfd8ed8812 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.362206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c57cc-8ec0-4242-a6fe-0adc9985ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.362841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c57cc-8ec0-4242-a6fe-0adc9985ee2e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.363167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c57cc-8ec0-4242-a6fe-0adc9985ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.363486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c57cc-8ec0-4242-a6fe-0adc9985ee2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.364180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c57cc-8ec0-4242-a6fe-0adc9985ee2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.366063 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c57cc-8ec0-4242-a6fe-0adc9985ee2e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.480898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1260/2539] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:27 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:27.492918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def907c7-9e61-44ec-bd3f-8c8ac014f901 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.504021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def907c7-9e61-44ec-bd3f-8c8ac014f901 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.562116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bec6d-4305-422b-b920-8437d412161f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.562816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bec6d-4305-422b-b920-8437d412161f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.563062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bec6d-4305-422b-b920-8437d412161f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.563449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bec6d-4305-422b-b920-8437d412161f admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.564137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bec6d-4305-422b-b920-8437d412161f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.566050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bec6d-4305-422b-b920-8437d412161f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.650171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1280/2540] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:27 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:27.653930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c57cc-8ec0-4242-a6fe-0adc9985ee2e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.672709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c57cc-8ec0-4242-a6fe-0adc9985ee2e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.679874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cec284-509c-4e3a-9643-e50197036020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.680565 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec284-509c-4e3a-9643-e50197036020 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.680878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec284-509c-4e3a-9643-e50197036020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.681155 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cec284-509c-4e3a-9643-e50197036020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.681936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cec284-509c-4e3a-9643-e50197036020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.683611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cec284-509c-4e3a-9643-e50197036020 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.705247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1261/2541] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:27.706630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bec6d-4305-422b-b920-8437d412161f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.718020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bec6d-4305-422b-b920-8437d412161f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.723023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7645bd-5260-477c-ab2c-a4696c4e804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.723950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7645bd-5260-477c-ab2c-a4696c4e804d admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.724237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7645bd-5260-477c-ab2c-a4696c4e804d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.724540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7645bd-5260-477c-ab2c-a4696c4e804d admin admin] PATH_INFO: `/v3/projects/9b1cdfd3cc104c2098cd8287c7c101a1/users/958a88da61784ef0b135287c84d2f5b5/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.739998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7645bd-5260-477c-ab2c-a4696c4e804d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1cdfd3cc104c2098cd8287c7c101a1, user_id=958a88da61784ef0b135287c84d2f5b5, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.742660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7645bd-5260-477c-ab2c-a4696c4e804d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.817750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1281/2542] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:27 2022] PUT /identity/v3/projects/9b1cdfd3cc104c2098cd8287c7c101a1/users/958a88da61784ef0b135287c84d2f5b5/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:27.823277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cec284-509c-4e3a-9643-e50197036020 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.834851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cec284-509c-4e3a-9643-e50197036020 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.858543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:27.901008 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84ae75-4401-4368-9ff0-8636ca4fee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.902378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84ae75-4401-4368-9ff0-8636ca4fee81 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.902653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84ae75-4401-4368-9ff0-8636ca4fee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.902939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84ae75-4401-4368-9ff0-8636ca4fee81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.903712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84ae75-4401-4368-9ff0-8636ca4fee81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.906281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84ae75-4401-4368-9ff0-8636ca4fee81 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.913873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7645bd-5260-477c-ab2c-a4696c4e804d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.925704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1262/2543] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:27.926575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84ae75-4401-4368-9ff0-8636ca4fee81 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:27.932457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7645bd-5260-477c-ab2c-a4696c4e804d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.948755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.950338 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.950835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.951683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] PATH_INFO: `/v3/users/545c418079c648d282b8bd9953ea3df0` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.952325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84ae75-4401-4368-9ff0-8636ca4fee81 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:27.953401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] RBAC: Authorizing `identity:delete_user(user_id=545c418079c648d282b8bd9953ea3df0)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.956381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.958572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dafe3-5a9c-4de8-89bb-380db13b4577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:27.960536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dafe3-5a9c-4de8-89bb-380db13b4577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:27.961336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dafe3-5a9c-4de8-89bb-380db13b4577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:27.961790 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dafe3-5a9c-4de8-89bb-380db13b4577 admin admin] PATH_INFO: `/v3/projects/9b1cdfd3cc104c2098cd8287c7c101a1/users/958a88da61784ef0b135287c84d2f5b5/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:27.965925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dafe3-5a9c-4de8-89bb-380db13b4577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1cdfd3cc104c2098cd8287c7c101a1, user_id=958a88da61784ef0b135287c84d2f5b5, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:27.968814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dafe3-5a9c-4de8-89bb-380db13b4577 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:27.988331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1263/2544] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:27 2022] PUT /identity/v3/projects/9b1cdfd3cc104c2098cd8287c7c101a1/users/958a88da61784ef0b135287c84d2f5b5/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:28.000608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084d175-eea9-45dc-899d-ce59fb99e4a9 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:28.000903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084d175-eea9-45dc-899d-ce59fb99e4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:28.001728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084d175-eea9-45dc-899d-ce59fb99e4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:28.031438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b084d175-eea9-45dc-899d-ce59fb99e4a9 None None] MFA Rules not processed for user `958a88da61784ef0b135287c84d2f5b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:28.032749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b084d175-eea9-45dc-899d-ce59fb99e4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:28.114666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1264/2545] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:28.126541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084d175-eea9-45dc-899d-ce59fb99e4a9 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:28.158644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '545c418079c648d282b8bd9953ea3df0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:28.159349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '545c418079c648d282b8bd9953ea3df0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:28.160060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '545c418079c648d282b8bd9953ea3df0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:28.173534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '545c418079c648d282b8bd9953ea3df0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:28.181521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:28.189111 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084d175-eea9-45dc-899d-ce59fb99e4a9 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:28.189311 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '545c418079c648d282b8bd9953ea3df0'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:28.198295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89fcd6c-69ad-47ff-b126-fb533ede1cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:28.198295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fcd6c-69ad-47ff-b126-fb533ede1cea service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:28.198295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fcd6c-69ad-47ff-b126-fb533ede1cea service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:28.198295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89fcd6c-69ad-47ff-b126-fb533ede1cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:28.198746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f89fcd6c-69ad-47ff-b126-fb533ede1cea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:28.242349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:28.256026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89fcd6c-69ad-47ff-b126-fb533ede1cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:28.257525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89fcd6c-69ad-47ff-b126-fb533ede1cea service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:28.267091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1265/2546] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:28.306459 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1282/2547] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:27 2022] DELETE /identity/v3/users/545c418079c648d282b8bd9953ea3df0 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:28.308238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba209ca6-fc04-4120-a296-ccef07ab3d2b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:28.308754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba209ca6-fc04-4120-a296-ccef07ab3d2b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:28.309161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba209ca6-fc04-4120-a296-ccef07ab3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:28.333431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ba209ca6-fc04-4120-a296-ccef07ab3d2b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:28.335362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ba209ca6-fc04-4120-a296-ccef07ab3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:28.409261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1266/2548] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:28.419162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:28.420718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:28.488461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be6ade-bc5b-4693-848f-4fc45d54deba admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:28.493802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6adf78-4e71-4b1c-ae15-e7406732ef50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:28.494503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6adf78-4e71-4b1c-ae15-e7406732ef50 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:28.494785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6adf78-4e71-4b1c-ae15-e7406732ef50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:28.495059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6adf78-4e71-4b1c-ae15-e7406732ef50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:28.505082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6adf78-4e71-4b1c-ae15-e7406732ef50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:28.506562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6adf78-4e71-4b1c-ae15-e7406732ef50 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:28.517939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1283/2549] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:28.913209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba209ca6-fc04-4120-a296-ccef07ab3d2b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:28.915933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ba209ca6-fc04-4120-a296-ccef07ab3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:28.929806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba209ca6-fc04-4120-a296-ccef07ab3d2b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:28.936108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:28.936108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:28.936108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:28.936108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] PATH_INFO: `/v3/projects/3f3e07278ebe462db9e1bbf84aee9bde` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:28.944960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3e07278ebe462db9e1bbf84aee9bde)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:28.944960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:29.086566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3e07278ebe462db9e1bbf84aee9bde', 'request_id': 'req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.094101 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3e07278ebe462db9e1bbf84aee9bde', 'request_id': 'req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.204293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3e07278ebe462db9e1bbf84aee9bde', 'request_id': 'req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.216742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] The token cache is being invalidate because project 3f3e07278ebe462db9e1bbf84aee9bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:29.217123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.217699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.219370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1267/2550] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:28 2022] DELETE /identity/v3/projects/3f3e07278ebe462db9e1bbf84aee9bde => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:29.223336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6adf78-4e71-4b1c-ae15-e7406732ef50 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:29.233034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6adf78-4e71-4b1c-ae15-e7406732ef50 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:29.292953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:29.294107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:29.294512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:29.294797 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] PATH_INFO: `/v3/users/78b7885809264084bdf438f27c79665e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:29.296162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b7885809264084bdf438f27c79665e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:29.298118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:29.503918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b7885809264084bdf438f27c79665e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.504547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b7885809264084bdf438f27c79665e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.505174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b7885809264084bdf438f27c79665e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.512731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b7885809264084bdf438f27c79665e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.525890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b7885809264084bdf438f27c79665e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:29.626722 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1284/2551] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:29 2022] DELETE /identity/v3/users/78b7885809264084bdf438f27c79665e => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:30.068133 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.077352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:30.092166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff6f4-1cbc-40c6-b7c7-b62c5fb9f22f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:30.161289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:30.162061 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.162346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.162619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] PATH_INFO: `/v3/projects/85352b6c150b4d0f86d76bca65de5148` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.163831 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85352b6c150b4d0f86d76bca65de5148)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:30.166340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:30.178361 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:30.199687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.201096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:30.206609 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:30.261546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a7e37-b720-461a-af64-8e0c960bdcde admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:30.279200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85352b6c150b4d0f86d76bca65de5148', 'request_id': 'req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.285477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85352b6c150b4d0f86d76bca65de5148', 'request_id': 'req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.312282 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf9fdb-8389-47c7-94be-292dc41e9577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:30.312938 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf9fdb-8389-47c7-94be-292dc41e9577 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.313227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf9fdb-8389-47c7-94be-292dc41e9577 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.313507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf9fdb-8389-47c7-94be-292dc41e9577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.315278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-9ebf9fdb-8389-47c7-94be-292dc41e9577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:30.347465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85352b6c150b4d0f86d76bca65de5148', 'request_id': 'req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.359086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] The token cache is being invalidate because project 85352b6c150b4d0f86d76bca65de5148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:30.359409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.360039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f9ec9d96-855c-473a-b3a7-6fd1ea61efd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.361640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1268/2552] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:30 2022] DELETE /identity/v3/projects/85352b6c150b4d0f86d76bca65de5148 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:30.366468 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b559b-050c-4f77-8d74-25f93586591e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.366761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b559b-050c-4f77-8d74-25f93586591e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.367065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b559b-050c-4f77-8d74-25f93586591e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.373501 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf9fdb-8389-47c7-94be-292dc41e9577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:30.375182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf9fdb-8389-47c7-94be-292dc41e9577 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:30.386930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1285/2553] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:30.387122 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8e9b559b-050c-4f77-8d74-25f93586591e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:30.388258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8e9b559b-050c-4f77-8d74-25f93586591e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:30.449158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1269/2554] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:30.456981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6a744-d9f1-49be-a332-4c27a4708487 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.457275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6a744-d9f1-49be-a332-4c27a4708487 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.457813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6a744-d9f1-49be-a332-4c27a4708487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.483091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6df6a744-d9f1-49be-a332-4c27a4708487 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:30.487827 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6df6a744-d9f1-49be-a332-4c27a4708487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:30.505855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1286/2555] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:30.507846 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6a744-d9f1-49be-a332-4c27a4708487 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.510551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b559b-050c-4f77-8d74-25f93586591e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.510551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8e9b559b-050c-4f77-8d74-25f93586591e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:30.518316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6a744-d9f1-49be-a332-4c27a4708487 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:30.523064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:30.523748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b559b-050c-4f77-8d74-25f93586591e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:30.523883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.524125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.524401 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.525714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:30.526042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:30.527654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:30.531376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:30.532498 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.532958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.533571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] PATH_INFO: `/v3/users/9064cd21f64d4110974d69b4ea94e58f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.535231 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9064cd21f64d4110974d69b4ea94e58f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:30.537458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:30.539579 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1287/2556] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:59:30.547653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.557425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ca5ce-1b1c-4fc5-b841-24c91d2b6132 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:30.562687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9f33a-fd77-46dd-a368-cd46d03a6ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:30.563633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9f33a-fd77-46dd-a368-cd46d03a6ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.563918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9f33a-fd77-46dd-a368-cd46d03a6ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.564195 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9f33a-fd77-46dd-a368-cd46d03a6ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.564882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9f33a-fd77-46dd-a368-cd46d03a6ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:30.566799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9f33a-fd77-46dd-a368-cd46d03a6ebd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:30.667544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1288/2557] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:30 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:30.671482 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9f33a-fd77-46dd-a368-cd46d03a6ebd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.736158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9f33a-fd77-46dd-a368-cd46d03a6ebd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:30.740992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:30.741660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.741945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.742238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.742907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:30.744897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:30.774362 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9064cd21f64d4110974d69b4ea94e58f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.775567 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9064cd21f64d4110974d69b4ea94e58f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.776169 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9064cd21f64d4110974d69b4ea94e58f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.784908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9064cd21f64d4110974d69b4ea94e58f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.800589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-745b0d01-1910-4655-a0ba-61e3f941957d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9064cd21f64d4110974d69b4ea94e58f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:30.847192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1289/2558] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:30 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:30.851186 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.853465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:30.883545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d731b9-9ec6-438c-bdc2-b9d695456796 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:30.888158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48d55-7f03-4137-909f-862dc9f89de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:30.888811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48d55-7f03-4137-909f-862dc9f89de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.889101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48d55-7f03-4137-909f-862dc9f89de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.889377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48d55-7f03-4137-909f-862dc9f89de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.890088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c48d55-7f03-4137-909f-862dc9f89de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:30.891696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c48d55-7f03-4137-909f-862dc9f89de3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:30.901601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1290/2559] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:30.905497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48d55-7f03-4137-909f-862dc9f89de3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.914818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48d55-7f03-4137-909f-862dc9f89de3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:30.920743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1270/2560] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:30 2022] DELETE /identity/v3/users/9064cd21f64d4110974d69b4ea94e58f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:30.920876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf16308-d97d-4a1b-a9ee-2c346f222356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:30.921010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf16308-d97d-4a1b-a9ee-2c346f222356 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.921271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf16308-d97d-4a1b-a9ee-2c346f222356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.921556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf16308-d97d-4a1b-a9ee-2c346f222356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.922647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf16308-d97d-4a1b-a9ee-2c346f222356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:30.923750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8c220-3a2d-44f8-890d-7e11df1fe687 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:30.924095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8c220-3a2d-44f8-890d-7e11df1fe687 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:30.924261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf16308-d97d-4a1b-a9ee-2c346f222356 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:30.924398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8c220-3a2d-44f8-890d-7e11df1fe687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:30.935735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1291/2561] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:30.939559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf16308-d97d-4a1b-a9ee-2c346f222356 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:30.944675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-13b8c220-3a2d-44f8-890d-7e11df1fe687 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:30.945897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-13b8c220-3a2d-44f8-890d-7e11df1fe687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:30.949706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf16308-d97d-4a1b-a9ee-2c346f222356 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:31.006977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1271/2562] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:31.007907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:31.008860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:31.009784 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:31.010117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] PATH_INFO: `/v3/projects/6729f91c6ade4487a54eae91e4a56cfa/users/10136c3edc8c451498d4064fb1603418/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:31.017557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8c220-3a2d-44f8-890d-7e11df1fe687 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:31.018978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-13b8c220-3a2d-44f8-890d-7e11df1fe687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:31.028149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6729f91c6ade4487a54eae91e4a56cfa, user_id=10136c3edc8c451498d4064fb1603418, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:31.031190 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:31.079959 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8c220-3a2d-44f8-890d-7e11df1fe687 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:31.083801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef51d3-e727-4bc8-81dd-08db5c939f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:31.084449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef51d3-e727-4bc8-81dd-08db5c939f23 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:31.084732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef51d3-e727-4bc8-81dd-08db5c939f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:31.085027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef51d3-e727-4bc8-81dd-08db5c939f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:31.094640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef51d3-e727-4bc8-81dd-08db5c939f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:31.096108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef51d3-e727-4bc8-81dd-08db5c939f23 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:31.106270 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1272/2563] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:31.118079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1292/2564] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:30 2022] PUT /identity/v3/projects/6729f91c6ade4487a54eae91e4a56cfa/users/10136c3edc8c451498d4064fb1603418/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:31.130887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b9b1-d9ca-4e2a-b501-a9ae8636715a None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:31.131811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b9b1-d9ca-4e2a-b501-a9ae8636715a None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:31.131811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994b9b1-d9ca-4e2a-b501-a9ae8636715a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:31.164088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-9994b9b1-d9ca-4e2a-b501-a9ae8636715a None None] MFA Rules not processed for user `10136c3edc8c451498d4064fb1603418`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:31.166344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9994b9b1-d9ca-4e2a-b501-a9ae8636715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:31.241344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1273/2565] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:31.284963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:31.293576 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:31.295422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:31.356387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d75e5-f9be-4c1a-b745-36dfeea680f1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:31.360238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e458f-c20f-43d6-9ed9-dcb37579d9ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:31.361012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e458f-c20f-43d6-9ed9-dcb37579d9ae service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:31.361221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e458f-c20f-43d6-9ed9-dcb37579d9ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:31.364102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e458f-c20f-43d6-9ed9-dcb37579d9ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:31.368646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:31.376755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e458f-c20f-43d6-9ed9-dcb37579d9ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:31.378263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e458f-c20f-43d6-9ed9-dcb37579d9ae service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:31.388162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1293/2566] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:31.598721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994b9b1-d9ca-4e2a-b501-a9ae8636715a None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:31.600173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-9994b9b1-d9ca-4e2a-b501-a9ae8636715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:31.667119 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994b9b1-d9ca-4e2a-b501-a9ae8636715a None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:31.671680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:31.672464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:31.672746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:31.673012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] PATH_INFO: `/v3/projects/63924816a0c242ca943117f422a7d778` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:31.674185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=63924816a0c242ca943117f422a7d778)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:31.676126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:31.788996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63924816a0c242ca943117f422a7d778', 'request_id': 'req-f2154815-6950-4a14-b621-305328139c0d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:31.794381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63924816a0c242ca943117f422a7d778', 'request_id': 'req-f2154815-6950-4a14-b621-305328139c0d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:31.850058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63924816a0c242ca943117f422a7d778', 'request_id': 'req-f2154815-6950-4a14-b621-305328139c0d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:31.862568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] The token cache is being invalidate because project 63924816a0c242ca943117f422a7d778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:31.862568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:31.863417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-f2154815-6950-4a14-b621-305328139c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:31.864998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1274/2567] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:31 2022] DELETE /identity/v3/projects/63924816a0c242ca943117f422a7d778 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:31.869850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721eeac-741c-4814-aba1-b86b52c032e5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:31.870204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721eeac-741c-4814-aba1-b86b52c032e5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:31.870494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721eeac-741c-4814-aba1-b86b52c032e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:31.888967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-8721eeac-741c-4814-aba1-b86b52c032e5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:31.892157 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8721eeac-741c-4814-aba1-b86b52c032e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:31.950200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1294/2568] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:32.008610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd77b8-edd5-4c53-a055-b3f574bb16a4 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.008947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd77b8-edd5-4c53-a055-b3f574bb16a4 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.009630 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd77b8-edd5-4c53-a055-b3f574bb16a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.029507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cbbd77b8-edd5-4c53-a055-b3f574bb16a4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:32.030826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cbbd77b8-edd5-4c53-a055-b3f574bb16a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.045056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:32.045556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1275/2569] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:32.048735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721eeac-741c-4814-aba1-b86b52c032e5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.063580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721eeac-741c-4814-aba1-b86b52c032e5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.076575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.077598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.077904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.078255 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.079142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:32.079545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.082809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.097958 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1295/2570] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:59:32.103134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd77b8-edd5-4c53-a055-b3f574bb16a4 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.113219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd77b8-edd5-4c53-a055-b3f574bb16a4 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.117988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.119232 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.119529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.119827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.121787 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.123970 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.210547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.225078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.225230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1276/2571] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:32 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:32.225230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.225230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.225683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3efc8-cce9-4e1f-ba73-59d4f31a439d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.278917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ea291-441c-4d11-b95f-1daef251868e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.282501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.283139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.283447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.283589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.283714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.284368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.284657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.284938 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] PATH_INFO: `/v3/users/1e97a1c95f2b4975b706c3f4328478ce` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.285529 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.286194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e97a1c95f2b4975b706c3f4328478ce)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.288189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.347438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.348863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.358810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1277/2572] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:32.362508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.369543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6958cc6-b6ec-4a7e-b97b-16c1e54f791a service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.374262 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d2d2f-83a6-42ea-b193-e82ae98efacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.374909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d2d2f-83a6-42ea-b193-e82ae98efacd admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.375223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d2d2f-83a6-42ea-b193-e82ae98efacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.375531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d2d2f-83a6-42ea-b193-e82ae98efacd admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.376222 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d2d2f-83a6-42ea-b193-e82ae98efacd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.378117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d2d2f-83a6-42ea-b193-e82ae98efacd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.382321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:32.481792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1278/2573] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:32 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:32.485712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d2d2f-83a6-42ea-b193-e82ae98efacd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.497545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e97a1c95f2b4975b706c3f4328478ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:32.498120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d2d2f-83a6-42ea-b193-e82ae98efacd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.500937 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e97a1c95f2b4975b706c3f4328478ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:32.501675 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e97a1c95f2b4975b706c3f4328478ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:32.503452 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.506314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.506314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.506314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.506314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.507080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.510943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e97a1c95f2b4975b706c3f4328478ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:32.521615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1279/2574] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:32.522492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.524039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b232f0c1-b1e9-49e2-b036-e816e67f1b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e97a1c95f2b4975b706c3f4328478ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:32.524257 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.538304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3dae4-a766-40ea-be70-83e425843973 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.544298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4783cc-f877-4a5e-a0a9-3f1cf19c7e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.544962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4783cc-f877-4a5e-a0a9-3f1cf19c7e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.545243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4783cc-f877-4a5e-a0a9-3f1cf19c7e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.545721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4783cc-f877-4a5e-a0a9-3f1cf19c7e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.546450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4783cc-f877-4a5e-a0a9-3f1cf19c7e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.548088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4783cc-f877-4a5e-a0a9-3f1cf19c7e5e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.560493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1280/2575] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:32.564689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4783cc-f877-4a5e-a0a9-3f1cf19c7e5e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.575363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4783cc-f877-4a5e-a0a9-3f1cf19c7e5e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.580075 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089c48c-7ea0-4fb2-85be-5bfc44d0a165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.580952 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089c48c-7ea0-4fb2-85be-5bfc44d0a165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.581355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089c48c-7ea0-4fb2-85be-5bfc44d0a165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.581643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089c48c-7ea0-4fb2-85be-5bfc44d0a165 admin admin] PATH_INFO: `/v3/projects/4dd5d54d4393442b995ef452ab412f9a/users/b5e6cbeac90e4f6baa5f216c81cb4f1a/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.595789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089c48c-7ea0-4fb2-85be-5bfc44d0a165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dd5d54d4393442b995ef452ab412f9a, user_id=b5e6cbeac90e4f6baa5f216c81cb4f1a, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.598474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089c48c-7ea0-4fb2-85be-5bfc44d0a165 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.640085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:32.645840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1296/2576] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:32 2022] DELETE /identity/v3/users/1e97a1c95f2b4975b706c3f4328478ce => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:32.651403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ec34-6243-4127-9330-e2811c2ddedf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.651777 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ec34-6243-4127-9330-e2811c2ddedf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.652130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8ec34-6243-4127-9330-e2811c2ddedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.670598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-82e8ec34-6243-4127-9330-e2811c2ddedf None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:32.671913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-82e8ec34-6243-4127-9330-e2811c2ddedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.702605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1281/2577] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:32 2022] PUT /identity/v3/projects/4dd5d54d4393442b995ef452ab412f9a/users/b5e6cbeac90e4f6baa5f216c81cb4f1a/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:32.712623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae3317-260d-4889-a762-8834b9467e89 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.712918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae3317-260d-4889-a762-8834b9467e89 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.713217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae3317-260d-4889-a762-8834b9467e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.738747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1297/2578] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:32.743048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8bae3317-260d-4889-a762-8834b9467e89 None None] MFA Rules not processed for user `b5e6cbeac90e4f6baa5f216c81cb4f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:32.744828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8bae3317-260d-4889-a762-8834b9467e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.749685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8ec34-6243-4127-9330-e2811c2ddedf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.751159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-82e8ec34-6243-4127-9330-e2811c2ddedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:32.793631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:32.813750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8ec34-6243-4127-9330-e2811c2ddedf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.822962 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1282/2579] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:32.827448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae3317-260d-4889-a762-8834b9467e89 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:32.866512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7119296-ae8d-47ac-b919-3c81b532255b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.867123 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7119296-ae8d-47ac-b919-3c81b532255b service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.867430 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7119296-ae8d-47ac-b919-3c81b532255b service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.867711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7119296-ae8d-47ac-b919-3c81b532255b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.890080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae3317-260d-4889-a762-8834b9467e89 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:32.894737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed9c75-9e6a-46fc-b79d-151baf8d5de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:32.895421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed9c75-9e6a-46fc-b79d-151baf8d5de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:32.895707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed9c75-9e6a-46fc-b79d-151baf8d5de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:32.895996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed9c75-9e6a-46fc-b79d-151baf8d5de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:32.896676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed9c75-9e6a-46fc-b79d-151baf8d5de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.898584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed9c75-9e6a-46fc-b79d-151baf8d5de4 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.928455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7119296-ae8d-47ac-b919-3c81b532255b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:32.929965 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7119296-ae8d-47ac-b919-3c81b532255b service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:32.940872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1298/2580] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:32.991759 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1283/2581] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:32 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:32.992933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7119296-ae8d-47ac-b919-3c81b532255b service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:33.002952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7119296-ae8d-47ac-b919-3c81b532255b service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:33.061376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5698aa-e7a7-4df8-9a73-bdce357b9bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:33.062048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5698aa-e7a7-4df8-9a73-bdce357b9bde admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.062333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5698aa-e7a7-4df8-9a73-bdce357b9bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.062613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5698aa-e7a7-4df8-9a73-bdce357b9bde admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.063342 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5698aa-e7a7-4df8-9a73-bdce357b9bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:33.065257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5698aa-e7a7-4df8-9a73-bdce357b9bde admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:33.175960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1299/2582] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:32 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:33.177149 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed9c75-9e6a-46fc-b79d-151baf8d5de4 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:33.188940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed9c75-9e6a-46fc-b79d-151baf8d5de4 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:33.194035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185269eb-1baf-43b7-b8ce-2fc64183f741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:33.194720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185269eb-1baf-43b7-b8ce-2fc64183f741 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.194988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185269eb-1baf-43b7-b8ce-2fc64183f741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.195340 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185269eb-1baf-43b7-b8ce-2fc64183f741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.197076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185269eb-1baf-43b7-b8ce-2fc64183f741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:33.198738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185269eb-1baf-43b7-b8ce-2fc64183f741 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:33.213710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1284/2583] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:33.215641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5698aa-e7a7-4df8-9a73-bdce357b9bde admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:33.226512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5698aa-e7a7-4df8-9a73-bdce357b9bde admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:33.231893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:33.232927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.236758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.236758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] PATH_INFO: `/v3/projects/dea9f28bcf53441382a1a171fb09ea60/users/56281f59e51942b298df04feb4973936/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.251294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea9f28bcf53441382a1a171fb09ea60, user_id=56281f59e51942b298df04feb4973936, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:33.254058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:33.319888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1300/2584] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:33 2022] PUT /identity/v3/projects/dea9f28bcf53441382a1a171fb09ea60/users/56281f59e51942b298df04feb4973936/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:33.324301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185269eb-1baf-43b7-b8ce-2fc64183f741 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:33.334785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185269eb-1baf-43b7-b8ce-2fc64183f741 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:33.397609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee6764-70ca-4223-a5d4-cbdcba052882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:33.397609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee6764-70ca-4223-a5d4-cbdcba052882 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.397609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee6764-70ca-4223-a5d4-cbdcba052882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.397609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee6764-70ca-4223-a5d4-cbdcba052882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.398005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee6764-70ca-4223-a5d4-cbdcba052882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:33.398005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee6764-70ca-4223-a5d4-cbdcba052882 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:33.399455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:33.401532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:33.411313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1285/2585] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:33.419934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee6764-70ca-4223-a5d4-cbdcba052882 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:33.422472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d5965-5027-4f74-a5d8-923574e721b5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:33.432610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:33.433812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.434417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.436617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] PATH_INFO: `/v3/projects/1dfda7e38cb746fa90fcedd54689f268` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.439174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dfda7e38cb746fa90fcedd54689f268)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:33.441257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:33.441698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee6764-70ca-4223-a5d4-cbdcba052882 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:33.453683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:33.461669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7035d9-0963-4b51-8bb0-6b7b2dc42ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:33.461669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7035d9-0963-4b51-8bb0-6b7b2dc42ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.463154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7035d9-0963-4b51-8bb0-6b7b2dc42ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.463154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7035d9-0963-4b51-8bb0-6b7b2dc42ee3 admin admin] PATH_INFO: `/v3/projects/dea9f28bcf53441382a1a171fb09ea60/users/56281f59e51942b298df04feb4973936/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.468367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7035d9-0963-4b51-8bb0-6b7b2dc42ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea9f28bcf53441382a1a171fb09ea60, user_id=56281f59e51942b298df04feb4973936, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:33.471025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7035d9-0963-4b51-8bb0-6b7b2dc42ee3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:33.487296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1286/2586] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:33 2022] PUT /identity/v3/projects/dea9f28bcf53441382a1a171fb09ea60/users/56281f59e51942b298df04feb4973936/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:33.495492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eb299-62c1-4739-8288-2ca08fd185db None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.496474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eb299-62c1-4739-8288-2ca08fd185db None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.496474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eb299-62c1-4739-8288-2ca08fd185db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.501818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:33.521955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-aa8eb299-62c1-4739-8288-2ca08fd185db None None] MFA Rules not processed for user `56281f59e51942b298df04feb4973936`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:33.523154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-aa8eb299-62c1-4739-8288-2ca08fd185db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:33.565187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dfda7e38cb746fa90fcedd54689f268', 'request_id': 'req-64017901-cd06-41a8-809d-0d7861ad45f1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:33.570668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dfda7e38cb746fa90fcedd54689f268', 'request_id': 'req-64017901-cd06-41a8-809d-0d7861ad45f1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:33.595011 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1287/2587] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:33.632945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eb299-62c1-4739-8288-2ca08fd185db None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:33.634984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-aa8eb299-62c1-4739-8288-2ca08fd185db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:33.645435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dfda7e38cb746fa90fcedd54689f268', 'request_id': 'req-64017901-cd06-41a8-809d-0d7861ad45f1'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:33.660439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] The token cache is being invalidate because project 1dfda7e38cb746fa90fcedd54689f268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:33.660768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:33.661445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:33.663202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1301/2588] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:33 2022] DELETE /identity/v3/projects/1dfda7e38cb746fa90fcedd54689f268 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:33.667104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:33.668503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:33.703393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eb299-62c1-4739-8288-2ca08fd185db None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:33.740071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:33.753050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:33.757773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64017901-cd06-41a8-809d-0d7861ad45f1 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:33.765018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:33.766140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.766866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.767196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] PATH_INFO: `/v3/users/e6f06650b9794da6a667f62b6ea13f44` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.768767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f06650b9794da6a667f62b6ea13f44)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:33.770729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:33.776465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:33.778283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:33.778283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:33.778283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:33.779936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:33.847421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:33.849132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:33.859606 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1288/2589] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:33.992566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f06650b9794da6a667f62b6ea13f44'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:33.993309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f06650b9794da6a667f62b6ea13f44'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:33.994348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f06650b9794da6a667f62b6ea13f44'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.001376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f06650b9794da6a667f62b6ea13f44'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.016818 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f06650b9794da6a667f62b6ea13f44'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.126485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1302/2590] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:33 2022] DELETE /identity/v3/users/e6f06650b9794da6a667f62b6ea13f44 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:34.138451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:34.139803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:34.151895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507eb0f-a639-4691-b833-9be29c533082 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:34.224668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:34.225423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:34.225782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:34.226126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:34.228002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:34.285789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:34.287234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:34.297014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1289/2591] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:34.572692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:34.574214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:34.643473 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d396c7-2618-4321-99e5-8f9a62c1b0ca admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:34.648289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:34.649298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:34.649584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:34.649855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] PATH_INFO: `/v3/projects/0dd9c94b0ed245cca318781930475c4e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:34.651039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd9c94b0ed245cca318781930475c4e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:34.652978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:34.660662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:34.665171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:34.676287 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6728775-4b8b-4f06-bdc2-da86426016c6 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:34.681470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:34.682335 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:34.682617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:34.682895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] PATH_INFO: `/v3/users/10136c3edc8c451498d4064fb1603418` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:34.684118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=10136c3edc8c451498d4064fb1603418)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:34.686025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:34.708322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:34.781346 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd9c94b0ed245cca318781930475c4e', 'request_id': 'req-7f0e3b1d-063f-4921-a502-65a0d1098fd7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.787306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd9c94b0ed245cca318781930475c4e', 'request_id': 'req-7f0e3b1d-063f-4921-a502-65a0d1098fd7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.867482 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd9c94b0ed245cca318781930475c4e', 'request_id': 'req-7f0e3b1d-063f-4921-a502-65a0d1098fd7'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.881053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] The token cache is being invalidate because project 0dd9c94b0ed245cca318781930475c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:34.881464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.882219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-7f0e3b1d-063f-4921-a502-65a0d1098fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.884995 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1303/2592] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:34 2022] DELETE /identity/v3/projects/0dd9c94b0ed245cca318781930475c4e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:34.889690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361143f-3ee2-4d74-87ba-0b6a43d6042b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:34.889977 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361143f-3ee2-4d74-87ba-0b6a43d6042b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:34.890260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361143f-3ee2-4d74-87ba-0b6a43d6042b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:34.890729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10136c3edc8c451498d4064fb1603418'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.891510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10136c3edc8c451498d4064fb1603418'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.892114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10136c3edc8c451498d4064fb1603418'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.900152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10136c3edc8c451498d4064fb1603418'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.903486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:34.910204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4361143f-3ee2-4d74-87ba-0b6a43d6042b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:34.911623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4361143f-3ee2-4d74-87ba-0b6a43d6042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:34.915114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a88b84-14c5-48b2-85be-79071da9e42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10136c3edc8c451498d4064fb1603418'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:34.971397 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1304/2593] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:34.979336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaea74-05f0-448f-b614-aaf3b29c050d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:34.979659 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaea74-05f0-448f-b614-aaf3b29c050d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:34.993159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaea74-05f0-448f-b614-aaf3b29c050d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:34.997555 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2abaea74-05f0-448f-b614-aaf3b29c050d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:34.998801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2abaea74-05f0-448f-b614-aaf3b29c050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:35.006978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1305/2594] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:35.011339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaea74-05f0-448f-b614-aaf3b29c050d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.020378 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaea74-05f0-448f-b614-aaf3b29c050d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.025826 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.026452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.026734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.027010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.027762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:35.028069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.029661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.041696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1306/2595] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:59:35.045664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.058528 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1290/2596] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:34 2022] DELETE /identity/v3/users/10136c3edc8c451498d4064fb1603418 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:35.060243 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb4020-e792-4ec0-8c25-9bb91633d88b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.064688 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d962472-f77a-4669-a046-54dccf7e93f3 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.064983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d962472-f77a-4669-a046-54dccf7e93f3 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.069128 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d962472-f77a-4669-a046-54dccf7e93f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.088685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8d962472-f77a-4669-a046-54dccf7e93f3 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:35.090020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8d962472-f77a-4669-a046-54dccf7e93f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:35.120559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ce3de-7811-41d6-b915-fe3a4fb23672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.121150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ce3de-7811-41d6-b915-fe3a4fb23672 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.121442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ce3de-7811-41d6-b915-fe3a4fb23672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.121726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ce3de-7811-41d6-b915-fe3a4fb23672 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.122425 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ce3de-7811-41d6-b915-fe3a4fb23672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.124418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ce3de-7811-41d6-b915-fe3a4fb23672 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.150339 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1291/2597] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:35.160027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d962472-f77a-4669-a046-54dccf7e93f3 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.161337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8d962472-f77a-4669-a046-54dccf7e93f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:35.216527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d962472-f77a-4669-a046-54dccf7e93f3 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.220247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f1cf2-ba53-4c2d-a9c7-409239c656fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.220878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f1cf2-ba53-4c2d-a9c7-409239c656fc service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.221182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f1cf2-ba53-4c2d-a9c7-409239c656fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.221444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f1cf2-ba53-4c2d-a9c7-409239c656fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.230295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f1cf2-ba53-4c2d-a9c7-409239c656fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.231757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f1cf2-ba53-4c2d-a9c7-409239c656fc service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.239858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1307/2598] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:35 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:35.241853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1292/2599] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:35.244259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ce3de-7811-41d6-b915-fe3a4fb23672 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.306821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ce3de-7811-41d6-b915-fe3a4fb23672 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.311642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23178c-3903-4b0b-a64f-fc39ec95c0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.312321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23178c-3903-4b0b-a64f-fc39ec95c0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.312613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23178c-3903-4b0b-a64f-fc39ec95c0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.312894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23178c-3903-4b0b-a64f-fc39ec95c0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.313629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23178c-3903-4b0b-a64f-fc39ec95c0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.315656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23178c-3903-4b0b-a64f-fc39ec95c0b5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.413456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1308/2600] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:35 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:35.417331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f1cf2-ba53-4c2d-a9c7-409239c656fc service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.427792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f1cf2-ba53-4c2d-a9c7-409239c656fc service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.433506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f5a7c-5e38-4382-bbce-841ab0bef84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.434210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5a7c-5e38-4382-bbce-841ab0bef84a admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.434496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5a7c-5e38-4382-bbce-841ab0bef84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.434845 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f5a7c-5e38-4382-bbce-841ab0bef84a admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.435610 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f5a7c-5e38-4382-bbce-841ab0bef84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.437215 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f5a7c-5e38-4382-bbce-841ab0bef84a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.449044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1293/2601] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:35.457162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23178c-3903-4b0b-a64f-fc39ec95c0b5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.470428 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23178c-3903-4b0b-a64f-fc39ec95c0b5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.480542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa39d35-9173-44e7-bef9-41695ce21008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.481334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa39d35-9173-44e7-bef9-41695ce21008 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.481689 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa39d35-9173-44e7-bef9-41695ce21008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.482124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa39d35-9173-44e7-bef9-41695ce21008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.482769 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa39d35-9173-44e7-bef9-41695ce21008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.484560 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa39d35-9173-44e7-bef9-41695ce21008 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.496314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1309/2602] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:35.500524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f5a7c-5e38-4382-bbce-841ab0bef84a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.512945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f5a7c-5e38-4382-bbce-841ab0bef84a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.517631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.518545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.518829 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.519102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] PATH_INFO: `/v3/projects/5fa30f2446f84844b9000f36089b673d/users/fcb119c663be41a48bf1a8fe23bcf661/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.534622 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa30f2446f84844b9000f36089b673d, user_id=fcb119c663be41a48bf1a8fe23bcf661, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.537459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.644987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:35.654107 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1294/2603] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:35 2022] PUT /identity/v3/projects/5fa30f2446f84844b9000f36089b673d/users/fcb119c663be41a48bf1a8fe23bcf661/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:35.664420 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90027d8d-1bd3-47bd-9ce4-6a4171f667c5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.664711 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90027d8d-1bd3-47bd-9ce4-6a4171f667c5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.665858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90027d8d-1bd3-47bd-9ce4-6a4171f667c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.675236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.679038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:35.694573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:35.700585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-90027d8d-1bd3-47bd-9ce4-6a4171f667c5 None None] MFA Rules not processed for user `fcb119c663be41a48bf1a8fe23bcf661`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:35.702890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-90027d8d-1bd3-47bd-9ce4-6a4171f667c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:35.755744 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45426d-4cdd-4f96-97d8-e64cfbc10863 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.760954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.761833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.762117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.762389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] PATH_INFO: `/v3/projects/6729f91c6ade4487a54eae91e4a56cfa` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.763602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6729f91c6ade4487a54eae91e4a56cfa)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.765477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.778395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1310/2604] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:35.794952 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90027d8d-1bd3-47bd-9ce4-6a4171f667c5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.818386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90027d8d-1bd3-47bd-9ce4-6a4171f667c5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.818386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9371a-6df0-4a32-8a2b-918f0fad9785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.818386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9371a-6df0-4a32-8a2b-918f0fad9785 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.818386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9371a-6df0-4a32-8a2b-918f0fad9785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.818845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9371a-6df0-4a32-8a2b-918f0fad9785 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.818845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9371a-6df0-4a32-8a2b-918f0fad9785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.818845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9371a-6df0-4a32-8a2b-918f0fad9785 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.888398 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1311/2605] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:35 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:35.888818 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6729f91c6ade4487a54eae91e4a56cfa', 'request_id': 'req-8225a494-6d41-40cc-9ee7-adbacae4436d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:35.890492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9371a-6df0-4a32-8a2b-918f0fad9785 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:35.897484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6729f91c6ade4487a54eae91e4a56cfa', 'request_id': 'req-8225a494-6d41-40cc-9ee7-adbacae4436d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:35.960656 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9371a-6df0-4a32-8a2b-918f0fad9785 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:35.966927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:35.967713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.967856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6729f91c6ade4487a54eae91e4a56cfa', 'request_id': 'req-8225a494-6d41-40cc-9ee7-adbacae4436d'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:35.968001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.968478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:35.969356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:35.971318 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:35.981171 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] The token cache is being invalidate because project 6729f91c6ade4487a54eae91e4a56cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:35.981450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:35.982135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8225a494-6d41-40cc-9ee7-adbacae4436d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:35.983812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1295/2606] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:35 2022] DELETE /identity/v3/projects/6729f91c6ade4487a54eae91e4a56cfa => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:35.988487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c190f6-03f9-4b68-8a46-4f9dd8a5bb23 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:35.988795 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c190f6-03f9-4b68-8a46-4f9dd8a5bb23 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:35.989076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c190f6-03f9-4b68-8a46-4f9dd8a5bb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.046653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-50c190f6-03f9-4b68-8a46-4f9dd8a5bb23 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:36.048027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-50c190f6-03f9-4b68-8a46-4f9dd8a5bb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:36.061901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1312/2607] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:35 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:36.062912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.065802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:36.079512 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1296/2608] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:36.083593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d754a4-95e8-4f09-8b35-10d1e1c9b68f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.091451 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a5270-dd9a-44e9-a01a-0ecb72ff851e None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.092003 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a5270-dd9a-44e9-a01a-0ecb72ff851e None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.092280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a5270-dd9a-44e9-a01a-0ecb72ff851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.094566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd9d8a-6afc-46b5-9040-fddb52cb787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.095398 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd9d8a-6afc-46b5-9040-fddb52cb787e admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.095692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd9d8a-6afc-46b5-9040-fddb52cb787e admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.096075 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd9d8a-6afc-46b5-9040-fddb52cb787e admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.096950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd9d8a-6afc-46b5-9040-fddb52cb787e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.098785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd9d8a-6afc-46b5-9040-fddb52cb787e admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.110584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1313/2609] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:36.114790 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd9d8a-6afc-46b5-9040-fddb52cb787e admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.116748 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-f57a5270-dd9a-44e9-a01a-0ecb72ff851e None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:36.118791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f57a5270-dd9a-44e9-a01a-0ecb72ff851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:36.132070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd9d8a-6afc-46b5-9040-fddb52cb787e admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.136059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1297/2610] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:36.138719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298b776-a6d3-4ab1-942c-6e667ae385bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.139814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b776-a6d3-4ab1-942c-6e667ae385bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.142092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b776-a6d3-4ab1-942c-6e667ae385bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.142092 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298b776-a6d3-4ab1-942c-6e667ae385bb admin admin] PATH_INFO: `/v3/projects/18f6471d09ab40f8bd5bbfd8c00071fd/users/b6a1f0d3082e4b90a9986f50dc59ab2b/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.143619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a5270-dd9a-44e9-a01a-0ecb72ff851e None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.153082 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57a5270-dd9a-44e9-a01a-0ecb72ff851e None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.158072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.158431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298b776-a6d3-4ab1-942c-6e667ae385bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f6471d09ab40f8bd5bbfd8c00071fd, user_id=b6a1f0d3082e4b90a9986f50dc59ab2b, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.158717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.159015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.159433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.160172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:36.160486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.161273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298b776-a6d3-4ab1-942c-6e667ae385bb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.162167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.174025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1298/2611] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 11:59:36.178408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.189660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e2b7c-119f-41fc-addf-5d479390ff8e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.193393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8d3d1-1892-4e17-9d0a-191042efbb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.194186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8d3d1-1892-4e17-9d0a-191042efbb63 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.194476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8d3d1-1892-4e17-9d0a-191042efbb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.194755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8d3d1-1892-4e17-9d0a-191042efbb63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.195493 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8d3d1-1892-4e17-9d0a-191042efbb63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.198352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8d3d1-1892-4e17-9d0a-191042efbb63 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.337036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1314/2612] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:36 2022] PUT /identity/v3/projects/18f6471d09ab40f8bd5bbfd8c00071fd/users/b6a1f0d3082e4b90a9986f50dc59ab2b/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:36.340879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298b776-a6d3-4ab1-942c-6e667ae385bb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.354865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:36.363864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1299/2613] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:36 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:36.367693 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8d3d1-1892-4e17-9d0a-191042efbb63 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.402755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298b776-a6d3-4ab1-942c-6e667ae385bb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.429943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8d3d1-1892-4e17-9d0a-191042efbb63 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.435226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874b196-8eb9-4f6d-a23d-af5cb786d9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.436633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874b196-8eb9-4f6d-a23d-af5cb786d9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.436633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874b196-8eb9-4f6d-a23d-af5cb786d9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.436633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874b196-8eb9-4f6d-a23d-af5cb786d9f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.437376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874b196-8eb9-4f6d-a23d-af5cb786d9f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.439284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874b196-8eb9-4f6d-a23d-af5cb786d9f3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.443318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:36.458877 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0bb4a-10f1-4895-ac12-a205f6607bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.459589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0bb4a-10f1-4895-ac12-a205f6607bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.459879 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0bb4a-10f1-4895-ac12-a205f6607bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.460169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0bb4a-10f1-4895-ac12-a205f6607bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.460902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0bb4a-10f1-4895-ac12-a205f6607bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.462521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0bb4a-10f1-4895-ac12-a205f6607bf3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.472864 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1315/2614] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:36.477101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0bb4a-10f1-4895-ac12-a205f6607bf3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.487322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0bb4a-10f1-4895-ac12-a205f6607bf3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.492233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8acad-f62a-45df-a9c5-e4d45f4c8115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.493660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8acad-f62a-45df-a9c5-e4d45f4c8115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.493949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8acad-f62a-45df-a9c5-e4d45f4c8115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.494225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8acad-f62a-45df-a9c5-e4d45f4c8115 admin admin] PATH_INFO: `/v3/projects/18f6471d09ab40f8bd5bbfd8c00071fd/users/b6a1f0d3082e4b90a9986f50dc59ab2b/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.496260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8acad-f62a-45df-a9c5-e4d45f4c8115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f6471d09ab40f8bd5bbfd8c00071fd, user_id=b6a1f0d3082e4b90a9986f50dc59ab2b, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.498983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8acad-f62a-45df-a9c5-e4d45f4c8115 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.508994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1316/2615] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:36 2022] PUT /identity/v3/projects/18f6471d09ab40f8bd5bbfd8c00071fd/users/b6a1f0d3082e4b90a9986f50dc59ab2b/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:36.519915 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30670884-16a7-491c-b519-50b7ced00e82 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.520211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30670884-16a7-491c-b519-50b7ced00e82 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.520498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30670884-16a7-491c-b519-50b7ced00e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.525277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1300/2616] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:36 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:36.530113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874b196-8eb9-4f6d-a23d-af5cb786d9f3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.540732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874b196-8eb9-4f6d-a23d-af5cb786d9f3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.548807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-30670884-16a7-491c-b519-50b7ced00e82 None None] MFA Rules not processed for user `b6a1f0d3082e4b90a9986f50dc59ab2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:36.550314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-30670884-16a7-491c-b519-50b7ced00e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:36.612814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189a8d2-2c0e-408f-adfd-90c864747e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.612814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189a8d2-2c0e-408f-adfd-90c864747e1c admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.612814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189a8d2-2c0e-408f-adfd-90c864747e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.612814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189a8d2-2c0e-408f-adfd-90c864747e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.613324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189a8d2-2c0e-408f-adfd-90c864747e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.613324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189a8d2-2c0e-408f-adfd-90c864747e1c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.623581 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1301/2617] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:36.625065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189a8d2-2c0e-408f-adfd-90c864747e1c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.629046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1317/2618] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:36.635768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189a8d2-2c0e-408f-adfd-90c864747e1c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.639358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30670884-16a7-491c-b519-50b7ced00e82 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.640721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1908-d478-45ef-996b-d5da74a33a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.641249 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-30670884-16a7-491c-b519-50b7ced00e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:36.641376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1908-d478-45ef-996b-d5da74a33a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.641666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1908-d478-45ef-996b-d5da74a33a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.641948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1908-d478-45ef-996b-d5da74a33a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.642720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e1908-d478-45ef-996b-d5da74a33a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.644363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e1908-d478-45ef-996b-d5da74a33a24 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.656269 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1302/2619] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 11:59:36.660375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1908-d478-45ef-996b-d5da74a33a24 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.677343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1908-d478-45ef-996b-d5da74a33a24 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.685329 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.687571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.687571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.687571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] PATH_INFO: `/v3/projects/0507d00afdc046a8bc4d1bd0fda1eb7b/users/431134cb94bd448cb6413da52ac4ed48/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.705192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0507d00afdc046a8bc4d1bd0fda1eb7b, user_id=431134cb94bd448cb6413da52ac4ed48, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.708338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.716175 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30670884-16a7-491c-b519-50b7ced00e82 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.717033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfa1c7-5b87-4687-add4-2113e3889b90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.717789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfa1c7-5b87-4687-add4-2113e3889b90 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.718138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfa1c7-5b87-4687-add4-2113e3889b90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.718486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfa1c7-5b87-4687-add4-2113e3889b90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.720629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-88cfa1c7-5b87-4687-add4-2113e3889b90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:36.779304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfa1c7-5b87-4687-add4-2113e3889b90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.780793 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfa1c7-5b87-4687-add4-2113e3889b90 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:36.789647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1318/2620] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:36.802221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1303/2621] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:36 2022] PUT /identity/v3/projects/0507d00afdc046a8bc4d1bd0fda1eb7b/users/431134cb94bd448cb6413da52ac4ed48/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:36.809817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1bc18-797f-496b-9243-8cda87ef32fd None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.810133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1bc18-797f-496b-9243-8cda87ef32fd None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.810414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1bc18-797f-496b-9243-8cda87ef32fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.837631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-46c1bc18-797f-496b-9243-8cda87ef32fd None None] MFA Rules not processed for user `431134cb94bd448cb6413da52ac4ed48`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:36.838835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-46c1bc18-797f-496b-9243-8cda87ef32fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:36.881359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:36.907831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1319/2622] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:36.919737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:36.921442 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:36.978145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165bb4-ce58-4043-b392-83ad33e1bfba admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:36.981950 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdb819-84b5-40d0-a7e6-ca41986ddb6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:36.982613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb819-84b5-40d0-a7e6-ca41986ddb6e service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:36.985559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb819-84b5-40d0-a7e6-ca41986ddb6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:36.985559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb819-84b5-40d0-a7e6-ca41986ddb6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:36.991471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdb819-84b5-40d0-a7e6-ca41986ddb6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:36.992966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdb819-84b5-40d0-a7e6-ca41986ddb6e service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:37.001549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1304/2623] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:37.729730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1bc18-797f-496b-9243-8cda87ef32fd None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:37.731379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-46c1bc18-797f-496b-9243-8cda87ef32fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:37.804222 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1bc18-797f-496b-9243-8cda87ef32fd None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:37.810542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:37.811582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:37.812982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:37.812982 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] PATH_INFO: `/v3/users/6d1d521bb1734f69978297470c4c1f40` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:37.813493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d1d521bb1734f69978297470c4c1f40)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:37.817773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:37.833570 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:38.102438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d1d521bb1734f69978297470c4c1f40'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:38.103104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d1d521bb1734f69978297470c4c1f40'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:38.103755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d1d521bb1734f69978297470c4c1f40'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:38.110391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d1d521bb1734f69978297470c4c1f40'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:38.122710 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d1d521bb1734f69978297470c4c1f40'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:38.238611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1320/2624] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:37 2022] DELETE /identity/v3/users/6d1d521bb1734f69978297470c4c1f40 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:38.243733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e311414-23da-4684-b462-3970cb2e5a1d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:38.244260 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e311414-23da-4684-b462-3970cb2e5a1d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:38.244373 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e311414-23da-4684-b462-3970cb2e5a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:38.250118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:38.262404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-1e311414-23da-4684-b462-3970cb2e5a1d None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:38.263635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1e311414-23da-4684-b462-3970cb2e5a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:38.319541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1305/2625] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:38.330647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:38.331989 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:38.390599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbd524-7bd8-4b6f-8410-3dabcb452c60 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:38.394415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f89a17-c0be-4dff-8f77-f69fcf5d80a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:38.395070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89a17-c0be-4dff-8f77-f69fcf5d80a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:38.395393 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89a17-c0be-4dff-8f77-f69fcf5d80a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:38.395678 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89a17-c0be-4dff-8f77-f69fcf5d80a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:38.405467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f89a17-c0be-4dff-8f77-f69fcf5d80a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:38.406900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f89a17-c0be-4dff-8f77-f69fcf5d80a5 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:38.417077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1321/2626] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:38.834112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e311414-23da-4684-b462-3970cb2e5a1d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:38.835815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1e311414-23da-4684-b462-3970cb2e5a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:38.850733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e311414-23da-4684-b462-3970cb2e5a1d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:38.859454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:38.859454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:38.859454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:38.859973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] PATH_INFO: `/v3/projects/17c996d4dd874a3cb918eb7fa229a75f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:38.861211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c996d4dd874a3cb918eb7fa229a75f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:38.864334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:38.912014 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9404354-400c-4382-ae13-2574d3b279b5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:38.917677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9404354-400c-4382-ae13-2574d3b279b5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:38.917677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9404354-400c-4382-ae13-2574d3b279b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:38.929753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d9404354-400c-4382-ae13-2574d3b279b5 None None] MFA Rules not processed for user `431134cb94bd448cb6413da52ac4ed48`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:38.931582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d9404354-400c-4382-ae13-2574d3b279b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:38.985715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c996d4dd874a3cb918eb7fa229a75f', 'request_id': 'req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:38.989781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1322/2627] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:38.991860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c996d4dd874a3cb918eb7fa229a75f', 'request_id': 'req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.109267 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c996d4dd874a3cb918eb7fa229a75f', 'request_id': 'req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.123378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] The token cache is being invalidate because project 17c996d4dd874a3cb918eb7fa229a75f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:39.123677 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.124365 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.126357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1306/2628] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:38 2022] DELETE /identity/v3/projects/17c996d4dd874a3cb918eb7fa229a75f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:39.130217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9404354-400c-4382-ae13-2574d3b279b5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:39.147571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9404354-400c-4382-ae13-2574d3b279b5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:39.196457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:39.216011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:39.216888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:39.217169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:39.217442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] PATH_INFO: `/v3/users/b4b671df72ac480aa062ea0df9ddda12` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:39.218634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b671df72ac480aa062ea0df9ddda12)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:39.220623 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:39.435171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b671df72ac480aa062ea0df9ddda12'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.435900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b671df72ac480aa062ea0df9ddda12'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.436508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b671df72ac480aa062ea0df9ddda12'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.443481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b671df72ac480aa062ea0df9ddda12'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.455869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b671df72ac480aa062ea0df9ddda12'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:39.549827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:39.551280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:39.565106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1323/2629] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:39 2022] DELETE /identity/v3/users/b4b671df72ac480aa062ea0df9ddda12 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 11:59:39.644441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faa0ea-8f1f-4f6b-ae22-ad0845c0ee86 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:39.672470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:39.707427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:39.708086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:39.708380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:39.708638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:39.710460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:39.722103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:39.723715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:39.734631 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1307/2630] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:40.017665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:40.019216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:40.095507 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bfac-73e8-4871-98d4-5fd65360d146 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:40.100319 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:40.101205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:40.101662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:40.102002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] PATH_INFO: `/v3/projects/aa90f8f77aa24825b4f1c1b785db3126` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:40.109118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa90f8f77aa24825b4f1c1b785db3126)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:40.109118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:40.250808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa90f8f77aa24825b4f1c1b785db3126', 'request_id': 'req-dcca9a6e-97fd-40fd-9f80-71ab99707a41'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.256325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa90f8f77aa24825b4f1c1b785db3126', 'request_id': 'req-dcca9a6e-97fd-40fd-9f80-71ab99707a41'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.258854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:40.351496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa90f8f77aa24825b4f1c1b785db3126', 'request_id': 'req-dcca9a6e-97fd-40fd-9f80-71ab99707a41'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.363716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] The token cache is being invalidate because project aa90f8f77aa24825b4f1c1b785db3126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:40.363913 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.364526 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.366065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1324/2631] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:40 2022] DELETE /identity/v3/projects/aa90f8f77aa24825b4f1c1b785db3126 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:40.370681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:40.381345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0593fe0-7d72-400a-ad5d-47b84192cf0c service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:40.439158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:40.440092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:40.440375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:40.440671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] PATH_INFO: `/v3/users/958a88da61784ef0b135287c84d2f5b5` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:40.441855 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=958a88da61784ef0b135287c84d2f5b5)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:40.443828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:40.644311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '958a88da61784ef0b135287c84d2f5b5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.645917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '958a88da61784ef0b135287c84d2f5b5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.646623 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '958a88da61784ef0b135287c84d2f5b5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.656258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '958a88da61784ef0b135287c84d2f5b5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.673870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-aed5d560-066d-438c-b7b0-140b860e85c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '958a88da61784ef0b135287c84d2f5b5'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:40.753219 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1308/2632] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:40 2022] DELETE /identity/v3/users/958a88da61784ef0b135287c84d2f5b5 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:41.165452 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:41.167008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:41.180097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca9a6e-97fd-40fd-9f80-71ab99707a41 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:41.237887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:41.238757 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:41.239043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:41.239347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] PATH_INFO: `/v3/projects/9b1cdfd3cc104c2098cd8287c7c101a1` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:41.240688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b1cdfd3cc104c2098cd8287c7c101a1)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:41.242649 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:41.346030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b1cdfd3cc104c2098cd8287c7c101a1', 'request_id': 'req-dc779dfe-513f-4496-98a7-f3330426a2a8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:41.351699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b1cdfd3cc104c2098cd8287c7c101a1', 'request_id': 'req-dc779dfe-513f-4496-98a7-f3330426a2a8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:41.437037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b1cdfd3cc104c2098cd8287c7c101a1', 'request_id': 'req-dc779dfe-513f-4496-98a7-f3330426a2a8'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:41.449541 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] The token cache is being invalidate because project 9b1cdfd3cc104c2098cd8287c7c101a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:41.451925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:41.451925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dc779dfe-513f-4496-98a7-f3330426a2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:41.452164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1325/2633] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:41 2022] DELETE /identity/v3/projects/9b1cdfd3cc104c2098cd8287c7c101a1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:41.457030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25ea4-3ec1-47dd-8ebd-1453d8405102 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:41.457319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25ea4-3ec1-47dd-8ebd-1453d8405102 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:41.457597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f25ea4-3ec1-47dd-8ebd-1453d8405102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:41.475138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e2f25ea4-3ec1-47dd-8ebd-1453d8405102 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:41.476345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e2f25ea4-3ec1-47dd-8ebd-1453d8405102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:41.516782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:41.544389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1309/2634] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:41.552116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97351ff-ce6d-4c87-977c-61d203135894 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:41.552417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97351ff-ce6d-4c87-977c-61d203135894 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:41.552726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97351ff-ce6d-4c87-977c-61d203135894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:41.572045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f97351ff-ce6d-4c87-977c-61d203135894 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:41.573266 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f97351ff-ce6d-4c87-977c-61d203135894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:41.581733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1326/2635] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:41.585861 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f25ea4-3ec1-47dd-8ebd-1453d8405102 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:41.595513 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f25ea4-3ec1-47dd-8ebd-1453d8405102 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:41.600247 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:41.600892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:41.601172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:41.601516 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:41.602956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:41.603239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:41.604852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:41.616755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1310/2636] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:59:41.621811 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97351ff-ce6d-4c87-977c-61d203135894 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:41.631519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97351ff-ce6d-4c87-977c-61d203135894 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:41.636709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a6d1c-6668-4ac4-a3e0-40252b81c48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:41.637356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a6d1c-6668-4ac4-a3e0-40252b81c48f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:41.637641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a6d1c-6668-4ac4-a3e0-40252b81c48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:41.637916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a6d1c-6668-4ac4-a3e0-40252b81c48f admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:41.638598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a6d1c-6668-4ac4-a3e0-40252b81c48f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:41.640564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a6d1c-6668-4ac4-a3e0-40252b81c48f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:41.753641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1327/2637] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:41 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:41.754749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:41.766261 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee6261-4cb1-46f8-8045-ad34cad34d48 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:41.825662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3f29-52ca-43c3-8451-d7defef7baf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:41.826328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3f29-52ca-43c3-8451-d7defef7baf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:41.826604 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3f29-52ca-43c3-8451-d7defef7baf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:41.826883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b3f29-52ca-43c3-8451-d7defef7baf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:41.827616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b3f29-52ca-43c3-8451-d7defef7baf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:41.829533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b3f29-52ca-43c3-8451-d7defef7baf7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:41.936308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1311/2638] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:41 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:41.940564 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a6d1c-6668-4ac4-a3e0-40252b81c48f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:41.951528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a6d1c-6668-4ac4-a3e0-40252b81c48f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:41.957437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398d0f4-98c9-4839-a4f7-0122679bca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:41.958099 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d0f4-98c9-4839-a4f7-0122679bca7a admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:41.958383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d0f4-98c9-4839-a4f7-0122679bca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:41.958718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d0f4-98c9-4839-a4f7-0122679bca7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:41.959543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398d0f4-98c9-4839-a4f7-0122679bca7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:41.961162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398d0f4-98c9-4839-a4f7-0122679bca7a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:41.971824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1328/2639] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:41.975943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3f29-52ca-43c3-8451-d7defef7baf7 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:41.986062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b3f29-52ca-43c3-8451-d7defef7baf7 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:41.990633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef52e3-0295-422c-abff-4998f89ed2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:41.991388 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef52e3-0295-422c-abff-4998f89ed2fd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:41.991674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef52e3-0295-422c-abff-4998f89ed2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:41.991947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef52e3-0295-422c-abff-4998f89ed2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:41.993240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef52e3-0295-422c-abff-4998f89ed2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:41.994826 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef52e3-0295-422c-abff-4998f89ed2fd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:42.004692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1312/2640] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:42.009386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398d0f4-98c9-4839-a4f7-0122679bca7a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:42.019520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398d0f4-98c9-4839-a4f7-0122679bca7a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:42.024416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:42.025313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:42.025594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:42.025898 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] PATH_INFO: `/v3/projects/beaa8c8fc42a415d8932b39399c01626/users/0e57e1f6aa3e4b679a8147e42b4666d4/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:42.040478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beaa8c8fc42a415d8932b39399c01626, user_id=0e57e1f6aa3e4b679a8147e42b4666d4, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:42.043153 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:42.215868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1329/2641] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:42 2022] PUT /identity/v3/projects/beaa8c8fc42a415d8932b39399c01626/users/0e57e1f6aa3e4b679a8147e42b4666d4/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:42.226734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac74d50-4567-4eb5-b470-b778204ba9e6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:42.227052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac74d50-4567-4eb5-b470-b778204ba9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:42.227397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac74d50-4567-4eb5-b470-b778204ba9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:42.254040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-7ac74d50-4567-4eb5-b470-b778204ba9e6 None None] MFA Rules not processed for user `0e57e1f6aa3e4b679a8147e42b4666d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:42.255245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-7ac74d50-4567-4eb5-b470-b778204ba9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:42.320309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1313/2642] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:42.349382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:42.351074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:42.370334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:42.410162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cc57b-4b6b-4da4-9034-133c8af0df76 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:42.413831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:42.414475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:42.414765 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:42.415044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:42.423260 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:42.424699 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:42.433542 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1330/2643] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 11:59:45.435751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac74d50-4567-4eb5-b470-b778204ba9e6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:45.446983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac74d50-4567-4eb5-b470-b778204ba9e6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:45.450746 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:45.454834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:45.454834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:45.454834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:45.454834 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:45.488259 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:45.528916 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:45.530314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:45.541274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1314/2644] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:45.792138 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:45.793854 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:45.857326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc429b-ae88-47a3-b70f-36e6ece011cf service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:45.861315 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:45.861968 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:45.862259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:45.862644 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:45.864593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:45.943627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:45.946559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:45.958775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1331/2645] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:46.099643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:46.108165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:46.121025 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7bfcc-7a7f-40d4-8e61-d2615f6b518e service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:46.137044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:46.137741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:46.138022 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:46.138319 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:46.140202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:46.153285 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:46.154782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:46.189558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1315/2646] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:46.878023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:46.880058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:46.894222 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447da93-72d7-4a02-b4f0-e05a12b752d9 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:46.900741 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:46.903097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:46.903619 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:46.904082 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] PATH_INFO: `/v3/users/0b8f21537fc14c6fabeec0252205e1dc` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:46.905610 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8f21537fc14c6fabeec0252205e1dc)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:46.908247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:47.083204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8f21537fc14c6fabeec0252205e1dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.084006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8f21537fc14c6fabeec0252205e1dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.084694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8f21537fc14c6fabeec0252205e1dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.093204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8f21537fc14c6fabeec0252205e1dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.099679 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:47.107273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-af6fdb50-9473-41d5-af47-05b897334610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8f21537fc14c6fabeec0252205e1dc'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.194657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1332/2647] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:46 2022] DELETE /identity/v3/users/0b8f21537fc14c6fabeec0252205e1dc => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:47.452791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:47.466574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bce71d-8ca6-401e-b9bf-2ff20c2016e6 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:47.536647 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:47.537751 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:47.538071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:47.538350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] PATH_INFO: `/v3/projects/679163b973694df7a7d0fb1184f31f7e` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:47.544125 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=679163b973694df7a7d0fb1184f31f7e)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:47.546067 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:47.699356 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '679163b973694df7a7d0fb1184f31f7e', 'request_id': 'req-5a13ffc8-9c60-494a-9262-c4eab2983da1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.705388 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '679163b973694df7a7d0fb1184f31f7e', 'request_id': 'req-5a13ffc8-9c60-494a-9262-c4eab2983da1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.781692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '679163b973694df7a7d0fb1184f31f7e', 'request_id': 'req-5a13ffc8-9c60-494a-9262-c4eab2983da1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.800874 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] The token cache is being invalidate because project 679163b973694df7a7d0fb1184f31f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 11:59:47.801168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.801936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-5a13ffc8-9c60-494a-9262-c4eab2983da1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 11:59:47.803611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1316/2648] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 11:59:47 2022] DELETE /identity/v3/projects/679163b973694df7a7d0fb1184f31f7e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:47.810513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece392-bb6b-4c39-8156-6b860284f3ca None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:47.810810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece392-bb6b-4c39-8156-6b860284f3ca None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:47.811116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ece392-bb6b-4c39-8156-6b860284f3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:47.830293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-82ece392-bb6b-4c39-8156-6b860284f3ca None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:47.831660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-82ece392-bb6b-4c39-8156-6b860284f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:47.891504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1333/2649] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:47.899697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1fb7-f34d-455a-807c-8e5b2472013f None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:47.899988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1fb7-f34d-455a-807c-8e5b2472013f None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:47.905753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1fb7-f34d-455a-807c-8e5b2472013f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:47.919212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-39de1fb7-f34d-455a-807c-8e5b2472013f None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:47.920447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-39de1fb7-f34d-455a-807c-8e5b2472013f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:47.928654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1317/2650] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 11:59:47.932921 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ece392-bb6b-4c39-8156-6b860284f3ca None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:47.942582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ece392-bb6b-4c39-8156-6b860284f3ca None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:47.947418 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:47.948071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:47.948358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:47.948643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:47.949370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 11:59:47.949688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:47.951381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:47.963135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1334/2651] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 11:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 11:59:47.967434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de1fb7-f34d-455a-807c-8e5b2472013f None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:47.976661 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de1fb7-f34d-455a-807c-8e5b2472013f None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:47.981898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d69e-b326-4968-9e07-e3f46f7e62e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:47.982535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d69e-b326-4968-9e07-e3f46f7e62e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:47.982814 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d69e-b326-4968-9e07-e3f46f7e62e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:47.983086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d69e-b326-4968-9e07-e3f46f7e62e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:47.983819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d69e-b326-4968-9e07-e3f46f7e62e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:47.985690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d69e-b326-4968-9e07-e3f46f7e62e9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:48.085699 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1318/2652] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:47 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 11:59:48.090005 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:48.100127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9322c5-eca3-4b93-902d-8a764a8bc4d4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:48.173135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bbdda-5545-4799-a229-4a5b8708827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:48.174095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bbdda-5545-4799-a229-4a5b8708827d admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:48.174531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bbdda-5545-4799-a229-4a5b8708827d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:48.174814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1bbdda-5545-4799-a229-4a5b8708827d admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:48.175796 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bbdda-5545-4799-a229-4a5b8708827d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:48.178302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1bbdda-5545-4799-a229-4a5b8708827d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:48.277137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1335/2653] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:48 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:48.281260 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d69e-b326-4968-9e07-e3f46f7e62e9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:48.292077 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d69e-b326-4968-9e07-e3f46f7e62e9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:48.296656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afee2d-470a-4ef7-be1c-6cc1d49b9f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:48.297303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afee2d-470a-4ef7-be1c-6cc1d49b9f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:48.297994 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afee2d-470a-4ef7-be1c-6cc1d49b9f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:48.298275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afee2d-470a-4ef7-be1c-6cc1d49b9f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:48.298996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afee2d-470a-4ef7-be1c-6cc1d49b9f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:48.300648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afee2d-470a-4ef7-be1c-6cc1d49b9f12 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:48.311757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1319/2654] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:48.315714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bbdda-5545-4799-a229-4a5b8708827d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:48.326121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1bbdda-5545-4799-a229-4a5b8708827d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:48.330788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd78c8b-4bfd-4601-a7cf-086f1b5b6ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:48.331512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd78c8b-4bfd-4601-a7cf-086f1b5b6ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:48.331840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd78c8b-4bfd-4601-a7cf-086f1b5b6ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:48.332158 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd78c8b-4bfd-4601-a7cf-086f1b5b6ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:48.332922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd78c8b-4bfd-4601-a7cf-086f1b5b6ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:48.334540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd78c8b-4bfd-4601-a7cf-086f1b5b6ecc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:48.344383 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1336/2655] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:48.348210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afee2d-470a-4ef7-be1c-6cc1d49b9f12 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:48.355002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:48.358644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afee2d-470a-4ef7-be1c-6cc1d49b9f12 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:48.363175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:48.364088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:48.364374 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:48.368906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] PATH_INFO: `/v3/projects/59ae388052ee406c8d6cc749a1ed8c02/users/fa171a9679a3460bb2ee9369e4051690/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:48.379226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ae388052ee406c8d6cc749a1ed8c02, user_id=fa171a9679a3460bb2ee9369e4051690, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:48.382012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:48.488414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1320/2656] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:48 2022] PUT /identity/v3/projects/59ae388052ee406c8d6cc749a1ed8c02/users/fa171a9679a3460bb2ee9369e4051690/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:48.493481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2de33-4fd1-41ae-907a-1c99b1264f69 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:48.501872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2de33-4fd1-41ae-907a-1c99b1264f69 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:48.501872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2de33-4fd1-41ae-907a-1c99b1264f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:48.513466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7dc2de33-4fd1-41ae-907a-1c99b1264f69 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:48.514668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7dc2de33-4fd1-41ae-907a-1c99b1264f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:48.554715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:48.574068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1337/2657] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:48.583344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:48.585052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:48.650741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fc19e-f2c0-47d6-aa73-4c9777c62c08 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:48.654535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5805862-5502-4556-a070-2e8a67f6a7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:48.656979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5805862-5502-4556-a070-2e8a67f6a7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:48.656979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5805862-5502-4556-a070-2e8a67f6a7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:48.656979 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5805862-5502-4556-a070-2e8a67f6a7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:48.665803 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5805862-5502-4556-a070-2e8a67f6a7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:48.667239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5805862-5502-4556-a070-2e8a67f6a7f8 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:48.677467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1321/2658] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:55.308341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e91f-6942-48fa-a62c-4c207e330c9d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:55.308890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e91f-6942-48fa-a62c-4c207e330c9d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:55.308890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283e91f-6942-48fa-a62c-4c207e330c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:55.336923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3283e91f-6942-48fa-a62c-4c207e330c9d None None] MFA Rules not processed for user `fa171a9679a3460bb2ee9369e4051690`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 11:59:55.338151 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3283e91f-6942-48fa-a62c-4c207e330c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:55.430635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1338/2659] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 11:59:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 11:59:55.435394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5805862-5502-4556-a070-2e8a67f6a7f8 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:55.451683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5805862-5502-4556-a070-2e8a67f6a7f8 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:55.456530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bccb0-75eb-47d4-9404-f2b6541fbe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:55.457132 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bccb0-75eb-47d4-9404-f2b6541fbe25 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:55.457416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bccb0-75eb-47d4-9404-f2b6541fbe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:55.457695 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bccb0-75eb-47d4-9404-f2b6541fbe25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:55.458389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bccb0-75eb-47d4-9404-f2b6541fbe25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:55.460389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bccb0-75eb-47d4-9404-f2b6541fbe25 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:55.599707 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1322/2660] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 11:59:55 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:55.603951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283e91f-6942-48fa-a62c-4c207e330c9d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:55.616235 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283e91f-6942-48fa-a62c-4c207e330c9d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:55.675372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f23e06-a956-4e5c-aa8b-ded171385e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:55.676071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f23e06-a956-4e5c-aa8b-ded171385e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:55.676358 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f23e06-a956-4e5c-aa8b-ded171385e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:55.676639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f23e06-a956-4e5c-aa8b-ded171385e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:55.677368 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f23e06-a956-4e5c-aa8b-ded171385e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:55.679339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f23e06-a956-4e5c-aa8b-ded171385e6f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:55.763601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1339/2661] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 11:59:55 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 11:59:55.773515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bccb0-75eb-47d4-9404-f2b6541fbe25 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:55.784809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bccb0-75eb-47d4-9404-f2b6541fbe25 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:55.789598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2519-9101-4e0a-b594-d5c74bd6ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:55.790283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2519-9101-4e0a-b594-d5c74bd6ac0c admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:55.790568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2519-9101-4e0a-b594-d5c74bd6ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:55.790850 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2519-9101-4e0a-b594-d5c74bd6ac0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:55.791635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d2519-9101-4e0a-b594-d5c74bd6ac0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:55.793245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d2519-9101-4e0a-b594-d5c74bd6ac0c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:55.810965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1323/2662] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:55.812851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f23e06-a956-4e5c-aa8b-ded171385e6f admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:55.826105 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f23e06-a956-4e5c-aa8b-ded171385e6f admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:55.833046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1094ed-3c5a-43c1-8c3d-fa9c827eabeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:55.834436 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1094ed-3c5a-43c1-8c3d-fa9c827eabeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:55.834866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1094ed-3c5a-43c1-8c3d-fa9c827eabeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:55.835316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1094ed-3c5a-43c1-8c3d-fa9c827eabeb admin admin] PATH_INFO: `/v3/projects/2010c56213b144dfac669fe9c4ff5ae0/users/dcaf550f17164636be11f4400fc9c753/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:55.872289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1094ed-3c5a-43c1-8c3d-fa9c827eabeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2010c56213b144dfac669fe9c4ff5ae0, user_id=dcaf550f17164636be11f4400fc9c753, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:55.876612 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1094ed-3c5a-43c1-8c3d-fa9c827eabeb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:55.990407 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1340/2663] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:55 2022] PUT /identity/v3/projects/2010c56213b144dfac669fe9c4ff5ae0/users/dcaf550f17164636be11f4400fc9c753/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:55.990520 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2519-9101-4e0a-b594-d5c74bd6ac0c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:56.002973 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d2519-9101-4e0a-b594-d5c74bd6ac0c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:56.069271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:56.069762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:56.070053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:56.070910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:56.071990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:56.073597 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:56.086657 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1324/2664] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 11:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 11:59:56.089580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1094ed-3c5a-43c1-8c3d-fa9c827eabeb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:56.100622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1094ed-3c5a-43c1-8c3d-fa9c827eabeb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:56.105760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:56.106683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:56.106966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:56.107245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] PATH_INFO: `/v3/projects/2010c56213b144dfac669fe9c4ff5ae0/users/dcaf550f17164636be11f4400fc9c753/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:56.109602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2010c56213b144dfac669fe9c4ff5ae0, user_id=dcaf550f17164636be11f4400fc9c753, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:56.112313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:56.125218 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1341/2665] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 11:59:56 2022] PUT /identity/v3/projects/2010c56213b144dfac669fe9c4ff5ae0/users/dcaf550f17164636be11f4400fc9c753/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 11:59:58.084858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:58.088806 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:58.203106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 11:59:58.271644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e8040-ada3-40f3-a5c2-705e9fd48d53 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 11:59:58.376487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05589abd-955e-41e6-8098-6320f1f9fd3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 11:59:58.377674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589abd-955e-41e6-8098-6320f1f9fd3e service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 11:59:58.378295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589abd-955e-41e6-8098-6320f1f9fd3e service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 11:59:58.378767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05589abd-955e-41e6-8098-6320f1f9fd3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 11:59:58.381000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-05589abd-955e-41e6-8098-6320f1f9fd3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 11:59:58.395348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05589abd-955e-41e6-8098-6320f1f9fd3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 11:59:58.397313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05589abd-955e-41e6-8098-6320f1f9fd3e service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 11:59:58.420960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1325/2666] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 11:59:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 11:59:59.982210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 11:59:59.984003 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:00.042660 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:00.102690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6e092-1a73-4f39-8931-1417d559c254 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:00.108419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:00.109475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:00.109873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:00.110296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] PATH_INFO: `/v3/users/0e57e1f6aa3e4b679a8147e42b4666d4` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:00.111755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e57e1f6aa3e4b679a8147e42b4666d4)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:00.113810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:00.345761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e57e1f6aa3e4b679a8147e42b4666d4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:00.346451 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e57e1f6aa3e4b679a8147e42b4666d4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:00.347069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e57e1f6aa3e4b679a8147e42b4666d4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:00.355171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e57e1f6aa3e4b679a8147e42b4666d4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:00.369462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e57e1f6aa3e4b679a8147e42b4666d4'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:00.463540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1342/2667] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 11:59:59 2022] DELETE /identity/v3/users/0e57e1f6aa3e4b679a8147e42b4666d4 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:00.466598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc1dfc-0591-4e54-8ad3-6ef0bfb2d31b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:00.467084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc1dfc-0591-4e54-8ad3-6ef0bfb2d31b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:00.467156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc1dfc-0591-4e54-8ad3-6ef0bfb2d31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:00.496173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-61bc1dfc-0591-4e54-8ad3-6ef0bfb2d31b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:00.497460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-61bc1dfc-0591-4e54-8ad3-6ef0bfb2d31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:00.602984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1326/2668] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:00.613223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:00.614693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:00.678136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561690f4-4895-4505-8859-37a2c8b75ab6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:00.682553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4dc87-ad01-4e23-a1a6-327912dc286a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:00.683216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4dc87-ad01-4e23-a1a6-327912dc286a service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:00.683531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4dc87-ad01-4e23-a1a6-327912dc286a service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:00.683808 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4dc87-ad01-4e23-a1a6-327912dc286a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:00.694665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4dc87-ad01-4e23-a1a6-327912dc286a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:00.697080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4dc87-ad01-4e23-a1a6-327912dc286a service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:00.708767 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1343/2669] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:01.251291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc1dfc-0591-4e54-8ad3-6ef0bfb2d31b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:01.252972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-61bc1dfc-0591-4e54-8ad3-6ef0bfb2d31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:01.266978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc1dfc-0591-4e54-8ad3-6ef0bfb2d31b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:01.272605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:01.274551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:01.274975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:01.275417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] PATH_INFO: `/v3/projects/beaa8c8fc42a415d8932b39399c01626` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:01.277086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=beaa8c8fc42a415d8932b39399c01626)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:01.279307 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:01.433265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beaa8c8fc42a415d8932b39399c01626', 'request_id': 'req-05006a5b-3c1d-49df-a033-547e16757e5a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:01.442958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:01.447413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beaa8c8fc42a415d8932b39399c01626', 'request_id': 'req-05006a5b-3c1d-49df-a033-547e16757e5a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:01.524456 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beaa8c8fc42a415d8932b39399c01626', 'request_id': 'req-05006a5b-3c1d-49df-a033-547e16757e5a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:01.538363 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] The token cache is being invalidate because project beaa8c8fc42a415d8932b39399c01626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:01.538788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:01.542170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:01.542170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1327/2670] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:01 2022] DELETE /identity/v3/projects/beaa8c8fc42a415d8932b39399c01626 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:03.226891 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b06f8-bf9a-4f3a-a888-0aecc13c52eb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:03.227403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b06f8-bf9a-4f3a-a888-0aecc13c52eb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:03.227611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b06f8-bf9a-4f3a-a888-0aecc13c52eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:03.271508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-600b06f8-bf9a-4f3a-a888-0aecc13c52eb None None] MFA Rules not processed for user `dcaf550f17164636be11f4400fc9c753`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:03.272928 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-600b06f8-bf9a-4f3a-a888-0aecc13c52eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:03.318713 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:03.354779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1344/2671] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:03.368679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:03.370289 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:03.482873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05006a5b-3c1d-49df-a033-547e16757e5a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:03.493236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400a125-b429-44ff-9d1a-86f092faff25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:03.494124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400a125-b429-44ff-9d1a-86f092faff25 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:03.494496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400a125-b429-44ff-9d1a-86f092faff25 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:03.494938 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400a125-b429-44ff-9d1a-86f092faff25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:03.499830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c400a125-b429-44ff-9d1a-86f092faff25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:03.592334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400a125-b429-44ff-9d1a-86f092faff25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:03.592334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400a125-b429-44ff-9d1a-86f092faff25 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:03.602981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1328/2672] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:06.401213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4bdfa-2c08-485b-ab42-b38683669e7c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:06.401715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4bdfa-2c08-485b-ab42-b38683669e7c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:06.401902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4bdfa-2c08-485b-ab42-b38683669e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:06.419534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-44a4bdfa-2c08-485b-ab42-b38683669e7c None None] MFA Rules not processed for user `fa171a9679a3460bb2ee9369e4051690`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:06.420728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-44a4bdfa-2c08-485b-ab42-b38683669e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:06.427647 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1345/2673] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:10.529359 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e40375-6919-461f-a287-18caacbbe485 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:10.529828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e40375-6919-461f-a287-18caacbbe485 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:10.529910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e40375-6919-461f-a287-18caacbbe485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:10.547987 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-98e40375-6919-461f-a287-18caacbbe485 None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:10.549191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-98e40375-6919-461f-a287-18caacbbe485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:10.610780 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1329/2674] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 12:00:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:11.116067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4bdfa-2c08-485b-ab42-b38683669e7c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:11.117562 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-44a4bdfa-2c08-485b-ab42-b38683669e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:11.189084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4bdfa-2c08-485b-ab42-b38683669e7c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:11.197872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6680c36-e59b-487b-9da9-3c283fdb302b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:11.198744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6680c36-e59b-487b-9da9-3c283fdb302b service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:11.199045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6680c36-e59b-487b-9da9-3c283fdb302b service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:11.199363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6680c36-e59b-487b-9da9-3c283fdb302b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:11.208961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6680c36-e59b-487b-9da9-3c283fdb302b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:11.210597 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6680c36-e59b-487b-9da9-3c283fdb302b service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:11.218825 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1346/2675] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 12:00:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 12:00:12.048750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa59da-987b-4c8a-bf54-a1f5a15882fc None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:12.049212 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa59da-987b-4c8a-bf54-a1f5a15882fc None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:12.049330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa59da-987b-4c8a-bf54-a1f5a15882fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:12.067571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-40fa59da-987b-4c8a-bf54-a1f5a15882fc None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:12.068784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-40fa59da-987b-4c8a-bf54-a1f5a15882fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:12.076325 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1330/2676] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 11 12:00:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:12.088983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6680c36-e59b-487b-9da9-3c283fdb302b service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:12.099842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6680c36-e59b-487b-9da9-3c283fdb302b service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:12.105414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c2e3-3b59-49e0-ae05-ed90ca0c2ac1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:12.106051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c2e3-3b59-49e0-ae05-ed90ca0c2ac1 service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:12.106336 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c2e3-3b59-49e0-ae05-ed90ca0c2ac1 service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:12.106611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541c2e3-3b59-49e0-ae05-ed90ca0c2ac1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:12.114929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541c2e3-3b59-49e0-ae05-ed90ca0c2ac1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:12.116532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541c2e3-3b59-49e0-ae05-ed90ca0c2ac1 service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:12.124677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1347/2677] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 12:00:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 12:00:12.717106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d09b7-370e-4e9a-90b0-3d82da531c14 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:12.717398 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d09b7-370e-4e9a-90b0-3d82da531c14 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:12.717681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d09b7-370e-4e9a-90b0-3d82da531c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:12.735216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5a5d09b7-370e-4e9a-90b0-3d82da531c14 None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:12.736527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5a5d09b7-370e-4e9a-90b0-3d82da531c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:12.744203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1331/2678] 10.222.0.26 () {58 vars in 1009 bytes} [Mon Jul 11 12:00:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:12.755935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c2e3-3b59-49e0-ae05-ed90ca0c2ac1 service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:12.766990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541c2e3-3b59-49e0-ae05-ed90ca0c2ac1 service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:12.771899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f13dc-35a6-4fde-a9b0-e966dd695831 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:12.772533 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f13dc-35a6-4fde-a9b0-e966dd695831 service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:12.772816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f13dc-35a6-4fde-a9b0-e966dd695831 service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:12.773089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f13dc-35a6-4fde-a9b0-e966dd695831 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:12.780899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f13dc-35a6-4fde-a9b0-e966dd695831 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:12.785638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f13dc-35a6-4fde-a9b0-e966dd695831 service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:12.790273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1348/2679] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 12:00:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 12:00:13.340302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e00d2-0aa6-4095-be78-7f9f78c12023 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:13.340742 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e00d2-0aa6-4095-be78-7f9f78c12023 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:13.340853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e00d2-0aa6-4095-be78-7f9f78c12023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:13.358429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ef0e00d2-0aa6-4095-be78-7f9f78c12023 None None] MFA Rules not processed for user `9d71050d3f36438d8b2d2cd4c537d3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:13.359640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ef0e00d2-0aa6-4095-be78-7f9f78c12023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:13.366686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1332/2680] 10.222.0.26 () {58 vars in 967 bytes} [Mon Jul 11 12:00:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:13.379280 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f13dc-35a6-4fde-a9b0-e966dd695831 service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:13.390304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f13dc-35a6-4fde-a9b0-e966dd695831 service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:13.395166 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7556ce4bf09d4113aec2037dd438cecb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:13.395851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:13.396137 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:13.396415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:13.404843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:13.406441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:13.414365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1349/2681] 10.222.0.26 () {60 vars in 1384 bytes} [Mon Jul 11 12:00:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 12:00:17.435760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e00d2-0aa6-4095-be78-7f9f78c12023 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:17.437176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ef0e00d2-0aa6-4095-be78-7f9f78c12023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:17.443224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:17.508146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e00d2-0aa6-4095-be78-7f9f78c12023 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:17.513771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:17.514422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:17.514706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:17.514983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:17.517001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:17.529539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:17.531123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:17.540895 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1333/2682] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:18.147635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:18.149070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.216456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9599efe-e4d9-49cf-9c2e-0a7c054671c4 service swift] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:18.221061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:18.225755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:18.225755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:18.225755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] PATH_INFO: `/v3/users/431134cb94bd448cb6413da52ac4ed48` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:18.225755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=431134cb94bd448cb6413da52ac4ed48)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:18.225755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:18.248616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:18.278420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:18.279989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.294047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8f441-f57f-4a70-af8b-af7e8b0fa60d service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:18.298712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:18.299618 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:18.299907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:18.300187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] PATH_INFO: `/v3/users/fcb119c663be41a48bf1a8fe23bcf661` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:18.301385 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb119c663be41a48bf1a8fe23bcf661)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:18.303392 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:18.468957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '431134cb94bd448cb6413da52ac4ed48'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.469614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '431134cb94bd448cb6413da52ac4ed48'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.470296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '431134cb94bd448cb6413da52ac4ed48'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.477594 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '431134cb94bd448cb6413da52ac4ed48'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.490641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f8a3b23b-c956-4059-b391-0db31b4f0e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '431134cb94bd448cb6413da52ac4ed48'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.532281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb119c663be41a48bf1a8fe23bcf661'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.532969 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb119c663be41a48bf1a8fe23bcf661'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.533588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb119c663be41a48bf1a8fe23bcf661'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.543718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb119c663be41a48bf1a8fe23bcf661'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.555851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b4dfa2ce-8450-4c3e-9dca-26c98994f870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb119c663be41a48bf1a8fe23bcf661'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:18.569238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1350/2683] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:18 2022] DELETE /identity/v3/users/431134cb94bd448cb6413da52ac4ed48 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:18.573802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da1091-110a-4321-a5ac-f630e672f707 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:18.574116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da1091-110a-4321-a5ac-f630e672f707 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:18.574403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da1091-110a-4321-a5ac-f630e672f707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:18.591934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-c9da1091-110a-4321-a5ac-f630e672f707 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:18.593210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c9da1091-110a-4321-a5ac-f630e672f707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.622721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1334/2684] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:18 2022] DELETE /identity/v3/users/fcb119c663be41a48bf1a8fe23bcf661 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:18.626675 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5303887-45e9-41f2-b73b-6c966236a430 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:18.627071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5303887-45e9-41f2-b73b-6c966236a430 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:18.627477 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5303887-45e9-41f2-b73b-6c966236a430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:18.648839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c5303887-45e9-41f2-b73b-6c966236a430 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:18.650118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c5303887-45e9-41f2-b73b-6c966236a430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.665026 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1351/2685] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:18.680324 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da1091-110a-4321-a5ac-f630e672f707 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:18.682001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-c9da1091-110a-4321-a5ac-f630e672f707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.722323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1335/2686] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:18.732298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5303887-45e9-41f2-b73b-6c966236a430 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:18.733808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c5303887-45e9-41f2-b73b-6c966236a430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:18.743825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:18.765809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da1091-110a-4321-a5ac-f630e672f707 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:18.808841 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5303887-45e9-41f2-b73b-6c966236a430 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:18.812715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:18.813417 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:18.813700 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:18.813978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:18.826060 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:18.827684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:18.832234 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:18.832892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:18.833176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:18.833464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:18.838318 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1336/2687] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:18.844467 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:18.845940 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:18.850258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:18.857558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1352/2688] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:19.114348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.115958 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:19.130553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874720-8c2b-4394-a7b4-3b337365a343 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:19.135772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:19.137423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.137709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.137989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] PATH_INFO: `/v3/projects/0507d00afdc046a8bc4d1bd0fda1eb7b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.139594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0507d00afdc046a8bc4d1bd0fda1eb7b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:19.141586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:19.262648 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0507d00afdc046a8bc4d1bd0fda1eb7b', 'request_id': 'req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.270231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.271893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:19.276991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0507d00afdc046a8bc4d1bd0fda1eb7b', 'request_id': 'req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.296037 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbd939-3f07-4364-8a5c-10eb0f6e3234 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:19.360181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:19.361058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.361347 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.361629 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] PATH_INFO: `/v3/projects/5fa30f2446f84844b9000f36089b673d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.362904 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa30f2446f84844b9000f36089b673d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:19.363617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0507d00afdc046a8bc4d1bd0fda1eb7b', 'request_id': 'req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.364916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:19.376411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] The token cache is being invalidate because project 0507d00afdc046a8bc4d1bd0fda1eb7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:19.376708 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.377381 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-b9dee9e6-efff-4bbe-9b67-b5f51c7ce054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.379042 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1337/2689] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:19 2022] DELETE /identity/v3/projects/0507d00afdc046a8bc4d1bd0fda1eb7b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:19.385434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19afd6-72b6-4c34-9986-9daa5f1832c6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.385684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19afd6-72b6-4c34-9986-9daa5f1832c6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.385968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19afd6-72b6-4c34-9986-9daa5f1832c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.404721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0c19afd6-72b6-4c34-9986-9daa5f1832c6 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:19.405956 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0c19afd6-72b6-4c34-9986-9daa5f1832c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:19.414384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1338/2690] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:19.422544 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8c27-8547-4ca8-a56f-21f771d327aa None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.422863 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8c27-8547-4ca8-a56f-21f771d327aa None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.423144 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8c27-8547-4ca8-a56f-21f771d327aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.441878 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ceeb8c27-8547-4ca8-a56f-21f771d327aa None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:19.443154 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ceeb8c27-8547-4ca8-a56f-21f771d327aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:19.453258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1339/2691] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:19.457281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb8c27-8547-4ca8-a56f-21f771d327aa None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.467575 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb8c27-8547-4ca8-a56f-21f771d327aa None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:19.472288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:19.472942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.473223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.473494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.474582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:00:19.474888 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:19.476691 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:19.492272 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1340/2692] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 12:00:19.493348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.503615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5e25e-2258-463a-a8f2-ca8ae44ff8c5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:19.510861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa30f2446f84844b9000f36089b673d', 'request_id': 'req-973a532c-9152-4716-93e9-ed92aebf2835'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.517388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa30f2446f84844b9000f36089b673d', 'request_id': 'req-973a532c-9152-4716-93e9-ed92aebf2835'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.568939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:19.569609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.569890 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.570165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.570860 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:19.573097 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:19.584561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:19.586020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa30f2446f84844b9000f36089b673d', 'request_id': 'req-973a532c-9152-4716-93e9-ed92aebf2835'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.599111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] The token cache is being invalidate because project 5fa30f2446f84844b9000f36089b673d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:19.599442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.600081 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.601607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1353/2693] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:19 2022] DELETE /identity/v3/projects/5fa30f2446f84844b9000f36089b673d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:19.605551 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.606858 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:19.618969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a532c-9152-4716-93e9-ed92aebf2835 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:19.624829 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:19.625821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.626204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.626486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] PATH_INFO: `/v3/users/b6a1f0d3082e4b90a9986f50dc59ab2b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.627726 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a1f0d3082e4b90a9986f50dc59ab2b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:19.629706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:19.725896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1341/2694] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:19 2022] POST /identity/v3/projects => generated 379 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:19.730434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.732173 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:19.801423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fbf98-4144-45a0-86f8-3ad38f173deb admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:19.805993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:19.806679 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.806967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.807334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.808007 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:19.810416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:19.897304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a1f0d3082e4b90a9986f50dc59ab2b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.897951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a1f0d3082e4b90a9986f50dc59ab2b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.898553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a1f0d3082e4b90a9986f50dc59ab2b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.905369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a1f0d3082e4b90a9986f50dc59ab2b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.918288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1342/2695] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:19 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 12:00:19.920672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-4ccff618-1bfb-4119-92b6-60c3bfb13c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a1f0d3082e4b90a9986f50dc59ab2b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:19.925073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.928275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:19.940750 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6854978-2af3-4c0e-a490-449547525f0a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:19.945295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956538-5a0b-4c8d-9077-ab37c40f2ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:19.946203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956538-5a0b-4c8d-9077-ab37c40f2ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.946483 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956538-5a0b-4c8d-9077-ab37c40f2ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.946758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14956538-5a0b-4c8d-9077-ab37c40f2ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.947508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14956538-5a0b-4c8d-9077-ab37c40f2ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:19.949098 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14956538-5a0b-4c8d-9077-ab37c40f2ec3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:19.959531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1343/2696] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 12:00:19.963301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956538-5a0b-4c8d-9077-ab37c40f2ec3 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:19.972972 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14956538-5a0b-4c8d-9077-ab37c40f2ec3 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:19.977652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36c1f-084d-4f4c-a595-797495506b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:19.978298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36c1f-084d-4f4c-a595-797495506b89 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:19.978586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36c1f-084d-4f4c-a595-797495506b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:19.993710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f36c1f-084d-4f4c-a595-797495506b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:19.993710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f36c1f-084d-4f4c-a595-797495506b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:19.993710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f36c1f-084d-4f4c-a595-797495506b89 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:19.993710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1344/2697] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 12:00:19.997225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36c1f-084d-4f4c-a595-797495506b89 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:20.007151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f36c1f-084d-4f4c-a595-797495506b89 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:20.011843 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:20.012758 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.013020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.013298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] PATH_INFO: `/v3/projects/79c8c84b3f704d528ea772607da56ddf/users/fccd3de89cd94e5fa4c488c16d993241/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.028089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c8c84b3f704d528ea772607da56ddf, user_id=fccd3de89cd94e5fa4c488c16d993241, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:20.030785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:20.037315 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:20.042047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1354/2698] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:19 2022] DELETE /identity/v3/users/b6a1f0d3082e4b90a9986f50dc59ab2b => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:20.111466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1345/2699] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:19 2022] PUT /identity/v3/projects/79c8c84b3f704d528ea772607da56ddf/users/fccd3de89cd94e5fa4c488c16d993241/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:20.116568 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ac25e-a5fd-4776-aa99-7c1df4d698a3 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.116876 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ac25e-a5fd-4776-aa99-7c1df4d698a3 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.117164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ac25e-a5fd-4776-aa99-7c1df4d698a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.139238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-3b2ac25e-a5fd-4776-aa99-7c1df4d698a3 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:20.140481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3b2ac25e-a5fd-4776-aa99-7c1df4d698a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:20.206615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1355/2700] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:20.216603 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:20.218301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:20.283000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffeda1-419b-4ffc-b332-c574e919a91d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:20.286815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7fd60a-a737-4429-a46c-c82db37090de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:20.287514 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fd60a-a737-4429-a46c-c82db37090de service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.287798 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fd60a-a737-4429-a46c-c82db37090de service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.288074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7fd60a-a737-4429-a46c-c82db37090de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.299027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fd60a-a737-4429-a46c-c82db37090de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:20.300509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7fd60a-a737-4429-a46c-c82db37090de service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:20.312112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1346/2701] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:20.510900 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ac25e-a5fd-4776-aa99-7c1df4d698a3 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:20.513018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3b2ac25e-a5fd-4776-aa99-7c1df4d698a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:20.541553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ac25e-a5fd-4776-aa99-7c1df4d698a3 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:20.560664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:20.562673 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.563117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.563599 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] PATH_INFO: `/v3/projects/18f6471d09ab40f8bd5bbfd8c00071fd` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.565088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f6471d09ab40f8bd5bbfd8c00071fd)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:20.567164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:20.692894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f6471d09ab40f8bd5bbfd8c00071fd', 'request_id': 'req-0b29e533-20c2-4776-ae06-3255ad6a82ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:20.699896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f6471d09ab40f8bd5bbfd8c00071fd', 'request_id': 'req-0b29e533-20c2-4776-ae06-3255ad6a82ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:20.761069 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f6471d09ab40f8bd5bbfd8c00071fd', 'request_id': 'req-0b29e533-20c2-4776-ae06-3255ad6a82ce'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:20.772716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] The token cache is being invalidate because project 18f6471d09ab40f8bd5bbfd8c00071fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:20.773012 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:20.773638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0b29e533-20c2-4776-ae06-3255ad6a82ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:20.775177 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1356/2702] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:20 2022] DELETE /identity/v3/projects/18f6471d09ab40f8bd5bbfd8c00071fd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:20.780242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d37d6-923e-4822-986d-099aedf6d32b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.780729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d37d6-923e-4822-986d-099aedf6d32b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.780838 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d37d6-923e-4822-986d-099aedf6d32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.800070 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-0e0d37d6-923e-4822-986d-099aedf6d32b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:20.801274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0e0d37d6-923e-4822-986d-099aedf6d32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:20.835882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:20.868104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1347/2703] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:20.877705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15192-f940-4371-a470-11f6f82359bc None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.878006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15192-f940-4371-a470-11f6f82359bc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.878327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15192-f940-4371-a470-11f6f82359bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.898118 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2dd15192-f940-4371-a470-11f6f82359bc None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:20.899300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2dd15192-f940-4371-a470-11f6f82359bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:20.909300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1357/2704] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:20.913752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d37d6-923e-4822-986d-099aedf6d32b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:20.924883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d37d6-923e-4822-986d-099aedf6d32b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:20.932729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:20.932729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.932729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.932729 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.933072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:00:20.933072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:20.934546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:20.948358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1348/2705] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:00:20.952523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15192-f940-4371-a470-11f6f82359bc None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:20.962755 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15192-f940-4371-a470-11f6f82359bc None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:20.968873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b368129-a4e1-4fbb-adbc-a56bed207ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:20.969529 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b368129-a4e1-4fbb-adbc-a56bed207ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:20.969809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b368129-a4e1-4fbb-adbc-a56bed207ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:20.970095 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b368129-a4e1-4fbb-adbc-a56bed207ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:20.970833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b368129-a4e1-4fbb-adbc-a56bed207ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:20.972739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b368129-a4e1-4fbb-adbc-a56bed207ddc admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:21.106887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1358/2706] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:20 2022] POST /identity/v3/projects => generated 389 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 12:00:21.108041 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:21.118129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768420d-e668-42af-9778-ffb2293d6edd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:21.181010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f776b-0861-40a1-98d8-d55b58f2754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:21.181689 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f776b-0861-40a1-98d8-d55b58f2754e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:21.181985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f776b-0861-40a1-98d8-d55b58f2754e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:21.182253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f776b-0861-40a1-98d8-d55b58f2754e admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:21.182965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f776b-0861-40a1-98d8-d55b58f2754e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:21.184957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f776b-0861-40a1-98d8-d55b58f2754e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:21.272545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1349/2707] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:21 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:21.276523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b368129-a4e1-4fbb-adbc-a56bed207ddc admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:21.288057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b368129-a4e1-4fbb-adbc-a56bed207ddc admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:21.293481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e2227-f6b3-40c6-8dfd-4c2dc2a48ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:21.294247 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e2227-f6b3-40c6-8dfd-4c2dc2a48ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:21.294519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e2227-f6b3-40c6-8dfd-4c2dc2a48ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:21.298257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e2227-f6b3-40c6-8dfd-4c2dc2a48ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:21.298257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e2227-f6b3-40c6-8dfd-4c2dc2a48ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:21.298257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e2227-f6b3-40c6-8dfd-4c2dc2a48ba4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:21.307700 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1359/2708] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:21.312096 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f776b-0861-40a1-98d8-d55b58f2754e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:21.322644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f776b-0861-40a1-98d8-d55b58f2754e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:21.327370 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bc71c-eaff-4516-a0a2-4b2d8b391cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:21.328034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc71c-eaff-4516-a0a2-4b2d8b391cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:21.328327 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc71c-eaff-4516-a0a2-4b2d8b391cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:21.328612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bc71c-eaff-4516-a0a2-4b2d8b391cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:21.329348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bc71c-eaff-4516-a0a2-4b2d8b391cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:21.330942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bc71c-eaff-4516-a0a2-4b2d8b391cbd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:21.342404 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1350/2709] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:21.346832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e2227-f6b3-40c6-8dfd-4c2dc2a48ba4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:21.357558 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e2227-f6b3-40c6-8dfd-4c2dc2a48ba4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:21.362279 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0614aa9-cee4-4e8c-9f60-294dcd175202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:21.363195 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614aa9-cee4-4e8c-9f60-294dcd175202 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:21.363534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614aa9-cee4-4e8c-9f60-294dcd175202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:21.363799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614aa9-cee4-4e8c-9f60-294dcd175202 admin admin] PATH_INFO: `/v3/projects/9cb700a31b17427482cc1c2def9e8832/users/4857136962f341d99b143ed7208621db/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:21.383273 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0614aa9-cee4-4e8c-9f60-294dcd175202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb700a31b17427482cc1c2def9e8832, user_id=4857136962f341d99b143ed7208621db, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:21.386017 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0614aa9-cee4-4e8c-9f60-294dcd175202 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:21.506503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1360/2710] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:21 2022] PUT /identity/v3/projects/9cb700a31b17427482cc1c2def9e8832/users/4857136962f341d99b143ed7208621db/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:21.518494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e7157-cb15-48d4-9334-23730a126a01 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:21.518835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e7157-cb15-48d4-9334-23730a126a01 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:21.519120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e7157-cb15-48d4-9334-23730a126a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:21.548176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-eb2e7157-cb15-48d4-9334-23730a126a01 None None] MFA Rules not processed for user `4857136962f341d99b143ed7208621db`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:21.549408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-eb2e7157-cb15-48d4-9334-23730a126a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:21.616027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1351/2711] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:21.620494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0614aa9-cee4-4e8c-9f60-294dcd175202 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:21.644039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:21.688676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0614aa9-cee4-4e8c-9f60-294dcd175202 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:21.694023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dc020-3314-478f-900e-2e9bd18c862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:21.694719 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc020-3314-478f-900e-2e9bd18c862c admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:21.695007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc020-3314-478f-900e-2e9bd18c862c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:21.695325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc020-3314-478f-900e-2e9bd18c862c admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:21.696287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dc020-3314-478f-900e-2e9bd18c862c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:21.698223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397dc020-3314-478f-900e-2e9bd18c862c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:21.843758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1361/2712] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:21 2022] POST /identity/v3/projects => generated 393 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:21.847913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e7157-cb15-48d4-9334-23730a126a01 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:21.859079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e7157-cb15-48d4-9334-23730a126a01 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:21.923767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:21.966868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe80ea-21e8-4614-961b-9fdbf259526b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:21.972560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe80ea-21e8-4614-961b-9fdbf259526b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:21.972560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe80ea-21e8-4614-961b-9fdbf259526b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:21.972560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe80ea-21e8-4614-961b-9fdbf259526b admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:21.972560 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe80ea-21e8-4614-961b-9fdbf259526b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:21.977446 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe80ea-21e8-4614-961b-9fdbf259526b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:22.096922 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1352/2713] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:21 2022] POST /identity/v3/users => generated 303 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:22.101111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dc020-3314-478f-900e-2e9bd18c862c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:22.117011 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397dc020-3314-478f-900e-2e9bd18c862c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:22.123438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f6c76-880c-4402-a9eb-2b073d61e5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:22.124301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f6c76-880c-4402-a9eb-2b073d61e5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.124593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f6c76-880c-4402-a9eb-2b073d61e5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.124927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f6c76-880c-4402-a9eb-2b073d61e5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.125671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f6c76-880c-4402-a9eb-2b073d61e5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:22.127483 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f6c76-880c-4402-a9eb-2b073d61e5a3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:22.143161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1362/2714] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:22.144541 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe80ea-21e8-4614-961b-9fdbf259526b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:22.155897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe80ea-21e8-4614-961b-9fdbf259526b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:22.160643 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408dcc3-633f-4a38-89e8-ec39a125e257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:22.161540 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408dcc3-633f-4a38-89e8-ec39a125e257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.161825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408dcc3-633f-4a38-89e8-ec39a125e257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.162103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408dcc3-633f-4a38-89e8-ec39a125e257 admin admin] PATH_INFO: `/v3/projects/64e144cb16374a3c8975f9571e112d9c/users/68cef2f26025470aae93d422d431e62b/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.178009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408dcc3-633f-4a38-89e8-ec39a125e257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e144cb16374a3c8975f9571e112d9c, user_id=68cef2f26025470aae93d422d431e62b, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:22.181492 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408dcc3-633f-4a38-89e8-ec39a125e257 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:22.275638 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1353/2715] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:22 2022] PUT /identity/v3/projects/64e144cb16374a3c8975f9571e112d9c/users/68cef2f26025470aae93d422d431e62b/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:22.280083 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f6c76-880c-4402-a9eb-2b073d61e5a3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:22.296522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f6c76-880c-4402-a9eb-2b073d61e5a3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:22.375841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073da9f-231c-4cb4-824a-a3ed96ee07ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:22.376548 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073da9f-231c-4cb4-824a-a3ed96ee07ea admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.376833 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073da9f-231c-4cb4-824a-a3ed96ee07ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.377111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073da9f-231c-4cb4-824a-a3ed96ee07ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.377871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073da9f-231c-4cb4-824a-a3ed96ee07ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:22.379631 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e073da9f-231c-4cb4-824a-a3ed96ee07ea admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:22.393948 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1363/2716] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:22.396351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408dcc3-633f-4a38-89e8-ec39a125e257 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:22.409739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408dcc3-633f-4a38-89e8-ec39a125e257 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:22.414756 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:22.415706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.415983 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.416256 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] PATH_INFO: `/v3/projects/64e144cb16374a3c8975f9571e112d9c/users/68cef2f26025470aae93d422d431e62b/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.419496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e144cb16374a3c8975f9571e112d9c, user_id=68cef2f26025470aae93d422d431e62b, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:22.422773 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:22.438556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1354/2717] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:22 2022] PUT /identity/v3/projects/64e144cb16374a3c8975f9571e112d9c/users/68cef2f26025470aae93d422d431e62b/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:22.449472 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aa9c0-2ddd-4173-b7c1-37205a173f1d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.455498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aa9c0-2ddd-4173-b7c1-37205a173f1d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.455498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aa9c0-2ddd-4173-b7c1-37205a173f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.483344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-7e2aa9c0-2ddd-4173-b7c1-37205a173f1d None None] MFA Rules not processed for user `68cef2f26025470aae93d422d431e62b`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:22.484636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7e2aa9c0-2ddd-4173-b7c1-37205a173f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:22.568688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1364/2718] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:22 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:22.580061 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:22.582313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:22.663187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac83cf-3f38-43f3-80bc-5d5ca6b1ecd8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:22.672482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bd29a-6973-4b8b-a55d-a70666b05f93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:22.672482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bd29a-6973-4b8b-a55d-a70666b05f93 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.672482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bd29a-6973-4b8b-a55d-a70666b05f93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.672482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bd29a-6973-4b8b-a55d-a70666b05f93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.680139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bd29a-6973-4b8b-a55d-a70666b05f93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:22.681611 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bd29a-6973-4b8b-a55d-a70666b05f93 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:22.692106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1355/2719] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:22.751226 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aa9c0-2ddd-4173-b7c1-37205a173f1d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:22.755290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7e2aa9c0-2ddd-4173-b7c1-37205a173f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:22.809476 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aa9c0-2ddd-4173-b7c1-37205a173f1d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:22.810766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:22.811521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:22.811819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:22.812101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:22.826000 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:22.878479 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:22.880058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:22.895203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1365/2720] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:26.086876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75ec05-5dd6-4d6f-8890-e42c6584384d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:26.087390 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75ec05-5dd6-4d6f-8890-e42c6584384d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:26.087558 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75ec05-5dd6-4d6f-8890-e42c6584384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:26.118633 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-ae75ec05-5dd6-4d6f-8890-e42c6584384d None None] MFA Rules not processed for user `fccd3de89cd94e5fa4c488c16d993241`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:26.120035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ae75ec05-5dd6-4d6f-8890-e42c6584384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:26.188821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1356/2721] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:26.193454 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:26.194890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:26.268151 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268e959-49b0-476a-bd0d-e749d59d4767 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:26.275667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a033a9-314c-4ff7-aacc-2e8b687e8a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:26.276351 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a033a9-314c-4ff7-aacc-2e8b687e8a32 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:26.276642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a033a9-314c-4ff7-aacc-2e8b687e8a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:26.276916 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a033a9-314c-4ff7-aacc-2e8b687e8a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:26.277622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a033a9-314c-4ff7-aacc-2e8b687e8a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:26.280277 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a033a9-314c-4ff7-aacc-2e8b687e8a32 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:26.397530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1366/2722] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:26 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:26.401598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75ec05-5dd6-4d6f-8890-e42c6584384d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:26.413002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75ec05-5dd6-4d6f-8890-e42c6584384d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:26.426711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:26.475301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de445cef-3793-46f4-a8c6-ede737a9e43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:26.475985 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de445cef-3793-46f4-a8c6-ede737a9e43e admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:26.476321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de445cef-3793-46f4-a8c6-ede737a9e43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:26.476599 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de445cef-3793-46f4-a8c6-ede737a9e43e admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:26.477330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de445cef-3793-46f4-a8c6-ede737a9e43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:26.479280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de445cef-3793-46f4-a8c6-ede737a9e43e admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:26.612330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1357/2723] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:26 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:26.613535 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a033a9-314c-4ff7-aacc-2e8b687e8a32 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:26.623899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a033a9-314c-4ff7-aacc-2e8b687e8a32 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:26.628520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d3722-ae2d-43d9-883a-fb1f3255a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:26.629192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d3722-ae2d-43d9-883a-fb1f3255a06d admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:26.629504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d3722-ae2d-43d9-883a-fb1f3255a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:26.629787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d3722-ae2d-43d9-883a-fb1f3255a06d admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:26.630544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d3722-ae2d-43d9-883a-fb1f3255a06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:26.632167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32d3722-ae2d-43d9-883a-fb1f3255a06d admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:26.643206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1367/2724] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 12:00:26.647234 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de445cef-3793-46f4-a8c6-ede737a9e43e admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:26.657887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de445cef-3793-46f4-a8c6-ede737a9e43e admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:26.666976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639c0d2-75ca-4459-8e9c-ac464891aed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:26.667907 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639c0d2-75ca-4459-8e9c-ac464891aed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:26.668192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639c0d2-75ca-4459-8e9c-ac464891aed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:26.668465 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639c0d2-75ca-4459-8e9c-ac464891aed2 admin admin] PATH_INFO: `/v3/projects/727f4d20c6ff48c5befe4ebab07a3c67/users/7209127017464bb48e2d45e168ebeacf/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:26.689479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639c0d2-75ca-4459-8e9c-ac464891aed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=727f4d20c6ff48c5befe4ebab07a3c67, user_id=7209127017464bb48e2d45e168ebeacf, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:26.692124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639c0d2-75ca-4459-8e9c-ac464891aed2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:26.783068 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1358/2725] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:26 2022] PUT /identity/v3/projects/727f4d20c6ff48c5befe4ebab07a3c67/users/7209127017464bb48e2d45e168ebeacf/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:26.786326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d3722-ae2d-43d9-883a-fb1f3255a06d admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:26.797628 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d3722-ae2d-43d9-883a-fb1f3255a06d admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:26.864488 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d42c8-82d7-4467-bbed-5491cf451dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:26.865171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d42c8-82d7-4467-bbed-5491cf451dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:26.865572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d42c8-82d7-4467-bbed-5491cf451dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:26.865875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d42c8-82d7-4467-bbed-5491cf451dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:26.866602 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d42c8-82d7-4467-bbed-5491cf451dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:26.868341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d42c8-82d7-4467-bbed-5491cf451dc8 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:26.882585 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1368/2726] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:26.883894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639c0d2-75ca-4459-8e9c-ac464891aed2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:26.897002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639c0d2-75ca-4459-8e9c-ac464891aed2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:26.904472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887b995-eb10-47b0-a2ef-f6a4e85d90d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:26.905609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887b995-eb10-47b0-a2ef-f6a4e85d90d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:26.906071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887b995-eb10-47b0-a2ef-f6a4e85d90d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:26.906548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887b995-eb10-47b0-a2ef-f6a4e85d90d9 admin admin] PATH_INFO: `/v3/projects/727f4d20c6ff48c5befe4ebab07a3c67/users/7209127017464bb48e2d45e168ebeacf/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:26.909221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887b995-eb10-47b0-a2ef-f6a4e85d90d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=727f4d20c6ff48c5befe4ebab07a3c67, user_id=7209127017464bb48e2d45e168ebeacf, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:26.913422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887b995-eb10-47b0-a2ef-f6a4e85d90d9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:26.925615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1359/2727] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:26 2022] PUT /identity/v3/projects/727f4d20c6ff48c5befe4ebab07a3c67/users/7209127017464bb48e2d45e168ebeacf/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:28.222880 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d42c8-82d7-4467-bbed-5491cf451dc8 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:28.239781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d42c8-82d7-4467-bbed-5491cf451dc8 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:28.272725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:28.317352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:28.318463 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:28.318756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:28.319132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] PATH_INFO: `/v3/users/4857136962f341d99b143ed7208621db` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:28.320419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4857136962f341d99b143ed7208621db)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:28.323021 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:28.570437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4857136962f341d99b143ed7208621db'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:28.572650 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4857136962f341d99b143ed7208621db'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:28.573752 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4857136962f341d99b143ed7208621db'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:28.584056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4857136962f341d99b143ed7208621db'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:28.600954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4857136962f341d99b143ed7208621db'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:28.702050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1369/2728] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:28 2022] DELETE /identity/v3/users/4857136962f341d99b143ed7208621db => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:28.703996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe55d-fd36-4662-a21d-4b446d681b65 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:28.704489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe55d-fd36-4662-a21d-4b446d681b65 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:28.704596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe55d-fd36-4662-a21d-4b446d681b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:28.743017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-22ebe55d-fd36-4662-a21d-4b446d681b65 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:28.747175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-22ebe55d-fd36-4662-a21d-4b446d681b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:28.852865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1360/2729] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:28.869381 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:28.871571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:28.972996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2a44-5c02-42bf-97cf-28efaa33a667 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:28.984632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d759605-0997-4a32-aacc-3513516ea843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:28.985327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d759605-0997-4a32-aacc-3513516ea843 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:28.985622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d759605-0997-4a32-aacc-3513516ea843 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:28.985896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d759605-0997-4a32-aacc-3513516ea843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:29.001892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d759605-0997-4a32-aacc-3513516ea843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:29.001892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d759605-0997-4a32-aacc-3513516ea843 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:29.016888 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1370/2730] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:29.542185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe55d-fd36-4662-a21d-4b446d681b65 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:29.543792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-22ebe55d-fd36-4662-a21d-4b446d681b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:29.562809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe55d-fd36-4662-a21d-4b446d681b65 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:29.568346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:29.569467 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:29.569977 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:29.570448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] PATH_INFO: `/v3/projects/9cb700a31b17427482cc1c2def9e8832` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:29.572587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb700a31b17427482cc1c2def9e8832)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:29.579342 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:29.744202 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb700a31b17427482cc1c2def9e8832', 'request_id': 'req-64bc838f-656d-4892-bb19-8677ab8626e1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:29.759652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb700a31b17427482cc1c2def9e8832', 'request_id': 'req-64bc838f-656d-4892-bb19-8677ab8626e1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:29.831894 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb700a31b17427482cc1c2def9e8832', 'request_id': 'req-64bc838f-656d-4892-bb19-8677ab8626e1'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:29.833466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:29.846435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] The token cache is being invalidate because project 9cb700a31b17427482cc1c2def9e8832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:29.846727 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:29.847494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:29.849123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1361/2731] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:29 2022] DELETE /identity/v3/projects/9cb700a31b17427482cc1c2def9e8832 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:29.853423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d759605-0997-4a32-aacc-3513516ea843 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:29.867188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d759605-0997-4a32-aacc-3513516ea843 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:29.939205 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:29.954150 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:29.955296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:29.955789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:29.956131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] PATH_INFO: `/v3/users/68cef2f26025470aae93d422d431e62b` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:29.957435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=68cef2f26025470aae93d422d431e62b)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:29.960340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:30.310550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68cef2f26025470aae93d422d431e62b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:30.312239 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68cef2f26025470aae93d422d431e62b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:30.313102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68cef2f26025470aae93d422d431e62b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:30.324079 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68cef2f26025470aae93d422d431e62b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:30.335018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-41f48bde-017c-4406-bdb2-1dea59057d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68cef2f26025470aae93d422d431e62b'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:30.414415 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1371/2732] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:29 2022] DELETE /identity/v3/users/68cef2f26025470aae93d422d431e62b => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:30.906917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:30.908372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:30.931089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc838f-656d-4892-bb19-8677ab8626e1 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:31.017802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:31.019358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.019358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.019777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] PATH_INFO: `/v3/projects/64e144cb16374a3c8975f9571e112d9c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.021253 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e144cb16374a3c8975f9571e112d9c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:31.025440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:31.184328 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e144cb16374a3c8975f9571e112d9c', 'request_id': 'req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:31.192136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e144cb16374a3c8975f9571e112d9c', 'request_id': 'req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:31.289538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e144cb16374a3c8975f9571e112d9c', 'request_id': 'req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:31.303153 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] The token cache is being invalidate because project 64e144cb16374a3c8975f9571e112d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:31.303609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:31.304463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a5b7d1a9-27dc-4f0b-85d6-bc8a020c89f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:31.306712 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1362/2733] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:30 2022] DELETE /identity/v3/projects/64e144cb16374a3c8975f9571e112d9c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:31.316763 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730dcf9-2826-4378-a39f-df83854f0bbf None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.317357 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730dcf9-2826-4378-a39f-df83854f0bbf None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.318016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730dcf9-2826-4378-a39f-df83854f0bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.341316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-a730dcf9-2826-4378-a39f-df83854f0bbf None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:31.343148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a730dcf9-2826-4378-a39f-df83854f0bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:31.423387 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1372/2734] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:31.429092 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc348ca-fb84-41ee-b1b5-9500e1b9dc6b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.429664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc348ca-fb84-41ee-b1b5-9500e1b9dc6b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.429763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc348ca-fb84-41ee-b1b5-9500e1b9dc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.450570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-efc348ca-fb84-41ee-b1b5-9500e1b9dc6b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:31.451847 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-efc348ca-fb84-41ee-b1b5-9500e1b9dc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:31.465156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1363/2735] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:31.466902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730dcf9-2826-4378-a39f-df83854f0bbf None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:31.479056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a730dcf9-2826-4378-a39f-df83854f0bbf None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:31.484086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:31.486117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.486117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.486117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.486117 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:00:31.486410 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:31.487997 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:31.503290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1373/2736] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:00:31.507323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc348ca-fb84-41ee-b1b5-9500e1b9dc6b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:31.520360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc348ca-fb84-41ee-b1b5-9500e1b9dc6b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:31.525617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a062caa-b076-4dc0-acbe-65d58efe1f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:31.526413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062caa-b076-4dc0-acbe-65d58efe1f3c admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.526697 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062caa-b076-4dc0-acbe-65d58efe1f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.526975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062caa-b076-4dc0-acbe-65d58efe1f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.527737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a062caa-b076-4dc0-acbe-65d58efe1f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:31.529640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a062caa-b076-4dc0-acbe-65d58efe1f3c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:31.658059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1364/2737] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:31 2022] POST /identity/v3/projects => generated 401 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:31.661496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:31.672786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102de832-d958-4b79-b44e-f0d9170aeb75 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:31.735278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:31.753169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032deee4-d50e-4fae-b45b-23d357d6cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:31.753860 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032deee4-d50e-4fae-b45b-23d357d6cd0b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.754149 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032deee4-d50e-4fae-b45b-23d357d6cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.754478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032deee4-d50e-4fae-b45b-23d357d6cd0b admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.755221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032deee4-d50e-4fae-b45b-23d357d6cd0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:31.757295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032deee4-d50e-4fae-b45b-23d357d6cd0b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:31.859953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1374/2738] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:31 2022] POST /identity/v3/users => generated 301 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:31.863791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a062caa-b076-4dc0-acbe-65d58efe1f3c admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:31.880218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a062caa-b076-4dc0-acbe-65d58efe1f3c admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:31.889670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf287d-ad19-412c-a205-1fcbe2c57c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:31.891160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf287d-ad19-412c-a205-1fcbe2c57c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.892210 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf287d-ad19-412c-a205-1fcbe2c57c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.893194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf287d-ad19-412c-a205-1fcbe2c57c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.895160 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf287d-ad19-412c-a205-1fcbe2c57c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:31.898146 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf287d-ad19-412c-a205-1fcbe2c57c92 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:31.910305 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1365/2739] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:31.914607 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032deee4-d50e-4fae-b45b-23d357d6cd0b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:31.925696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032deee4-d50e-4fae-b45b-23d357d6cd0b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:31.930613 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6ff54-0d90-448e-9f2d-6ad637563496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:31.931356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ff54-0d90-448e-9f2d-6ad637563496 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.931648 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ff54-0d90-448e-9f2d-6ad637563496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.931929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6ff54-0d90-448e-9f2d-6ad637563496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.932666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6ff54-0d90-448e-9f2d-6ad637563496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:31.935291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6ff54-0d90-448e-9f2d-6ad637563496 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:31.948417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1375/2740] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:31.953185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf287d-ad19-412c-a205-1fcbe2c57c92 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:31.965034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf287d-ad19-412c-a205-1fcbe2c57c92 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:31.970486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ebe63-7ba6-460e-a922-4787c5682d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:31.971433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ebe63-7ba6-460e-a922-4787c5682d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:31.971719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ebe63-7ba6-460e-a922-4787c5682d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:31.971996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ebe63-7ba6-460e-a922-4787c5682d0b admin admin] PATH_INFO: `/v3/projects/2c165a933868449fa4872dad9b3cb1c2/users/9b30ca943dd845cbb45c8d302f35777f/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:31.989853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ebe63-7ba6-460e-a922-4787c5682d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c165a933868449fa4872dad9b3cb1c2, user_id=9b30ca943dd845cbb45c8d302f35777f, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:31.993768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ebe63-7ba6-460e-a922-4787c5682d0b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:32.087966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1366/2741] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:31 2022] PUT /identity/v3/projects/2c165a933868449fa4872dad9b3cb1c2/users/9b30ca943dd845cbb45c8d302f35777f/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:32.096929 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238bec4-f799-4e50-933b-cf03b5759767 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:32.097439 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238bec4-f799-4e50-933b-cf03b5759767 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:32.097723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238bec4-f799-4e50-933b-cf03b5759767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:32.135443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-d238bec4-f799-4e50-933b-cf03b5759767 None None] MFA Rules not processed for user `9b30ca943dd845cbb45c8d302f35777f`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:32.137187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d238bec4-f799-4e50-933b-cf03b5759767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:32.237730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1376/2742] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:32.240583 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ebe63-7ba6-460e-a922-4787c5682d0b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:32.276405 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:32.336720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ebe63-7ba6-460e-a922-4787c5682d0b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:32.346000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441ba46-ed81-4804-ae11-c37ae70d62e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:32.346704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441ba46-ed81-4804-ae11-c37ae70d62e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:32.346988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441ba46-ed81-4804-ae11-c37ae70d62e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:32.347293 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441ba46-ed81-4804-ae11-c37ae70d62e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:32.347996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441ba46-ed81-4804-ae11-c37ae70d62e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:32.349990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441ba46-ed81-4804-ae11-c37ae70d62e2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:32.457424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1367/2743] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:32 2022] POST /identity/v3/projects => generated 401 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:32.461363 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238bec4-f799-4e50-933b-cf03b5759767 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:32.472810 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238bec4-f799-4e50-933b-cf03b5759767 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:32.540634 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cb3f2-e2bb-4e14-a019-6e1b597f55bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:32.546067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cb3f2-e2bb-4e14-a019-6e1b597f55bd admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:32.546067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cb3f2-e2bb-4e14-a019-6e1b597f55bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:32.546067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cb3f2-e2bb-4e14-a019-6e1b597f55bd admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:32.546067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cb3f2-e2bb-4e14-a019-6e1b597f55bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:32.546067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cb3f2-e2bb-4e14-a019-6e1b597f55bd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:32.636027 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1377/2744] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:32 2022] POST /identity/v3/users => generated 307 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 12:00:32.640621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441ba46-ed81-4804-ae11-c37ae70d62e2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:32.659785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441ba46-ed81-4804-ae11-c37ae70d62e2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:32.665218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c40107-e69f-4940-8428-61bd9773adaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:32.665360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c40107-e69f-4940-8428-61bd9773adaf admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:32.665634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c40107-e69f-4940-8428-61bd9773adaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:32.665905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c40107-e69f-4940-8428-61bd9773adaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:32.666644 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c40107-e69f-4940-8428-61bd9773adaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:32.668314 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c40107-e69f-4940-8428-61bd9773adaf admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:32.682871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1368/2745] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:32.687371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cb3f2-e2bb-4e14-a019-6e1b597f55bd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:32.708032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cb3f2-e2bb-4e14-a019-6e1b597f55bd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:32.723822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523df52d-d58d-47fc-8435-a0ad58c59754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:32.731633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523df52d-d58d-47fc-8435-a0ad58c59754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:32.731633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523df52d-d58d-47fc-8435-a0ad58c59754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:32.731633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523df52d-d58d-47fc-8435-a0ad58c59754 admin admin] PATH_INFO: `/v3/projects/4e0fc9ee0dcf4b9e8f1df879108f913f/users/724e64eb334e4dc1a3b24ab0fd8836cd/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:32.772855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523df52d-d58d-47fc-8435-a0ad58c59754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0fc9ee0dcf4b9e8f1df879108f913f, user_id=724e64eb334e4dc1a3b24ab0fd8836cd, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:32.776717 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523df52d-d58d-47fc-8435-a0ad58c59754 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:32.873291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1378/2746] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:32 2022] PUT /identity/v3/projects/4e0fc9ee0dcf4b9e8f1df879108f913f/users/724e64eb334e4dc1a3b24ab0fd8836cd/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:32.880296 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c40107-e69f-4940-8428-61bd9773adaf admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:32.895704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c40107-e69f-4940-8428-61bd9773adaf admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:32.995981 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a30ea10-3350-4003-b6f4-6ca06e443613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:32.997592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30ea10-3350-4003-b6f4-6ca06e443613 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:32.997875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30ea10-3350-4003-b6f4-6ca06e443613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:32.998971 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a30ea10-3350-4003-b6f4-6ca06e443613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:33.001220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30ea10-3350-4003-b6f4-6ca06e443613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:33.002767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a30ea10-3350-4003-b6f4-6ca06e443613 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:33.019302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1369/2747] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:33.021892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523df52d-d58d-47fc-8435-a0ad58c59754 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:33.046553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523df52d-d58d-47fc-8435-a0ad58c59754 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:33.053312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43db95-2620-4c87-bc88-a30122435f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:33.054301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43db95-2620-4c87-bc88-a30122435f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:33.054593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43db95-2620-4c87-bc88-a30122435f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:33.054877 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43db95-2620-4c87-bc88-a30122435f22 admin admin] PATH_INFO: `/v3/projects/4e0fc9ee0dcf4b9e8f1df879108f913f/users/724e64eb334e4dc1a3b24ab0fd8836cd/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:33.057753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43db95-2620-4c87-bc88-a30122435f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0fc9ee0dcf4b9e8f1df879108f913f, user_id=724e64eb334e4dc1a3b24ab0fd8836cd, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:33.062395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43db95-2620-4c87-bc88-a30122435f22 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:33.097216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1379/2748] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:33 2022] PUT /identity/v3/projects/4e0fc9ee0dcf4b9e8f1df879108f913f/users/724e64eb334e4dc1a3b24ab0fd8836cd/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:33.098345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17bfa7-3ec0-4e3f-9d3e-c2f2096b85ec None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:33.098736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17bfa7-3ec0-4e3f-9d3e-c2f2096b85ec None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:33.099016 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17bfa7-3ec0-4e3f-9d3e-c2f2096b85ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:33.129764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86fbea-52d8-4020-b4fa-4749bfe90deb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:33.130417 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86fbea-52d8-4020-b4fa-4749bfe90deb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:33.130930 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86fbea-52d8-4020-b4fa-4749bfe90deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:33.216721 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-2f17bfa7-3ec0-4e3f-9d3e-c2f2096b85ec None None] MFA Rules not processed for user `724e64eb334e4dc1a3b24ab0fd8836cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:33.218108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2f17bfa7-3ec0-4e3f-9d3e-c2f2096b85ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:33.222194 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:33.233625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ca86fbea-52d8-4020-b4fa-4749bfe90deb None None] MFA Rules not processed for user `7209127017464bb48e2d45e168ebeacf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:33.235074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ca86fbea-52d8-4020-b4fa-4749bfe90deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:33.346359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1380/2749] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:33.346359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86fbea-52d8-4020-b4fa-4749bfe90deb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:33.353458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ca86fbea-52d8-4020-b4fa-4749bfe90deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:33.397263 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1370/2750] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:33.400891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17bfa7-3ec0-4e3f-9d3e-c2f2096b85ec None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:33.408227 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2f17bfa7-3ec0-4e3f-9d3e-c2f2096b85ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:33.447910 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:33.466538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86fbea-52d8-4020-b4fa-4749bfe90deb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:33.472598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:33.473361 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:33.473715 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:33.474139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:33.476362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:33.489271 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:33.490980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:33.501290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1381/2751] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:33.508553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:33.510403 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:33.522238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17bfa7-3ec0-4e3f-9d3e-c2f2096b85ec None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:33.532479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:33.533753 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:33.534435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:33.534799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:33.537114 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:33.593712 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636735c-b1f2-4309-a107-74eab0cc32b3 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:33.604422 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:33.605091 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:33.606171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:33.606478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:33.619976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:33.621503 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:33.634776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1382/2752] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:33.643545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:33.646322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:33.662168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1371/2753] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:34.009615 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:34.011030 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:34.038831 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b8606-d9cf-4d1b-92a0-067cdc1acf50 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:34.043035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31a3b1-aa58-4317-8ad3-caf5fcb5013c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:34.043832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a3b1-aa58-4317-8ad3-caf5fcb5013c service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:34.044116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a3b1-aa58-4317-8ad3-caf5fcb5013c service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:34.044392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31a3b1-aa58-4317-8ad3-caf5fcb5013c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:34.097456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:34.112996 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31a3b1-aa58-4317-8ad3-caf5fcb5013c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:34.114355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31a3b1-aa58-4317-8ad3-caf5fcb5013c service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:34.124442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1383/2754] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:35.824867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:35.826496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:35.917681 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c9402-9760-4d04-9bd1-c3c8ac058adb service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:35.922998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:35.924065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:35.924355 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:35.924703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] PATH_INFO: `/v3/users/b5e6cbeac90e4f6baa5f216c81cb4f1a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:35.925926 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5e6cbeac90e4f6baa5f216c81cb4f1a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:35.927884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:36.165812 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5e6cbeac90e4f6baa5f216c81cb4f1a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:36.166475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5e6cbeac90e4f6baa5f216c81cb4f1a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:36.167582 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5e6cbeac90e4f6baa5f216c81cb4f1a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:36.174383 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:36.177602 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5e6cbeac90e4f6baa5f216c81cb4f1a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:36.190393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5e6cbeac90e4f6baa5f216c81cb4f1a'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:36.279781 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1372/2755] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:35 2022] DELETE /identity/v3/users/b5e6cbeac90e4f6baa5f216c81cb4f1a => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:36.284487 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507676-85d3-478d-ba56-0516f1c7bd0b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:36.284947 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507676-85d3-478d-ba56-0516f1c7bd0b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:36.285265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c507676-85d3-478d-ba56-0516f1c7bd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:36.303611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6c507676-85d3-478d-ba56-0516f1c7bd0b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:36.304835 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6c507676-85d3-478d-ba56-0516f1c7bd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:36.370224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1384/2756] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:36.387470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:36.389073 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:36.450352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4c51d-eed3-40be-b694-3d62fff0861a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:36.454006 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d572dd-c376-44d8-b629-6a2a53383439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:36.454663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d572dd-c376-44d8-b629-6a2a53383439 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:36.454947 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d572dd-c376-44d8-b629-6a2a53383439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:36.455433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d572dd-c376-44d8-b629-6a2a53383439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:36.465585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d572dd-c376-44d8-b629-6a2a53383439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:36.467038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d572dd-c376-44d8-b629-6a2a53383439 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:36.477810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1373/2757] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:37.040893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c507676-85d3-478d-ba56-0516f1c7bd0b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:37.040893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6c507676-85d3-478d-ba56-0516f1c7bd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:37.092298 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c507676-85d3-478d-ba56-0516f1c7bd0b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:37.110073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:37.111584 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:37.112110 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:37.112617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] PATH_INFO: `/v3/projects/4dd5d54d4393442b995ef452ab412f9a` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:37.120663 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dd5d54d4393442b995ef452ab412f9a)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:37.133897 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:37.297165 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dd5d54d4393442b995ef452ab412f9a', 'request_id': 'req-b804b10b-2e2a-4f87-8266-c0809c29c8da'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.302676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dd5d54d4393442b995ef452ab412f9a', 'request_id': 'req-b804b10b-2e2a-4f87-8266-c0809c29c8da'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.381257 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dd5d54d4393442b995ef452ab412f9a', 'request_id': 'req-b804b10b-2e2a-4f87-8266-c0809c29c8da'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.403007 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] The token cache is being invalidate because project 4dd5d54d4393442b995ef452ab412f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:37.403353 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.404020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.408552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1385/2758] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:37 2022] DELETE /identity/v3/projects/4dd5d54d4393442b995ef452ab412f9a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:37.409853 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d572dd-c376-44d8-b629-6a2a53383439 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:37.422112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d572dd-c376-44d8-b629-6a2a53383439 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:37.480141 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:37.482048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:37.482489 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:37.482767 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] PATH_INFO: `/v3/users/56281f59e51942b298df04feb4973936` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:37.484009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56281f59e51942b298df04feb4973936)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:37.486384 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:37.712519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56281f59e51942b298df04feb4973936'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.713246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56281f59e51942b298df04feb4973936'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.713927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56281f59e51942b298df04feb4973936'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.737275 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56281f59e51942b298df04feb4973936'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.754103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-32fcfc0e-5dd4-4c85-a34a-62a877e8a747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56281f59e51942b298df04feb4973936'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:37.763448 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:37.877497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1374/2759] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:37 2022] DELETE /identity/v3/users/56281f59e51942b298df04feb4973936 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:38.635466 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:38.638735 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:38.688341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804b10b-2e2a-4f87-8266-c0809c29c8da admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:38.794399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:38.795339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:38.795600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:38.795883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] PATH_INFO: `/v3/projects/dea9f28bcf53441382a1a171fb09ea60` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:38.797109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea9f28bcf53441382a1a171fb09ea60)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:38.799031 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:38.821530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:39.000468 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea9f28bcf53441382a1a171fb09ea60', 'request_id': 'req-f9026ea5-d0df-4776-aaf7-4773a73e3461'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:39.014630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea9f28bcf53441382a1a171fb09ea60', 'request_id': 'req-f9026ea5-d0df-4776-aaf7-4773a73e3461'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:39.117688 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea9f28bcf53441382a1a171fb09ea60', 'request_id': 'req-f9026ea5-d0df-4776-aaf7-4773a73e3461'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:39.132534 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] The token cache is being invalidate because project dea9f28bcf53441382a1a171fb09ea60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:39.133288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:39.136220 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f9026ea5-d0df-4776-aaf7-4773a73e3461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:39.140869 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1386/2760] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:38 2022] DELETE /identity/v3/projects/dea9f28bcf53441382a1a171fb09ea60 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:39.146135 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96484-ce65-4cf0-8b23-79e8ccb3077c None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.146711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96484-ce65-4cf0-8b23-79e8ccb3077c None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.146711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96484-ce65-4cf0-8b23-79e8ccb3077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.212719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-e8e96484-ce65-4cf0-8b23-79e8ccb3077c None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:39.213986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-e8e96484-ce65-4cf0-8b23-79e8ccb3077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:39.335425 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1375/2761] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:39.336056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688df2-6c36-403b-984f-b588ec0e8132 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.336360 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688df2-6c36-403b-984f-b588ec0e8132 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.336646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b688df2-6c36-403b-984f-b588ec0e8132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.358245 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-5b688df2-6c36-403b-984f-b588ec0e8132 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:39.359580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-5b688df2-6c36-403b-984f-b588ec0e8132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:39.382878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1387/2762] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:39.388851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96484-ce65-4cf0-8b23-79e8ccb3077c None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:39.399568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96484-ce65-4cf0-8b23-79e8ccb3077c None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:39.404854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:39.405667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.405951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.406228 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.407377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:00:39.407763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:39.409587 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:39.429789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1376/2763] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:00:39.434701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688df2-6c36-403b-984f-b588ec0e8132 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:39.449728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b688df2-6c36-403b-984f-b588ec0e8132 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:39.454851 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f316e2-af80-49ad-920d-1c20f4103630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:39.458421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f316e2-af80-49ad-920d-1c20f4103630 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.458421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f316e2-af80-49ad-920d-1c20f4103630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.458421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f316e2-af80-49ad-920d-1c20f4103630 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.458421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f316e2-af80-49ad-920d-1c20f4103630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:39.458746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f316e2-af80-49ad-920d-1c20f4103630 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:39.579349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1388/2764] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:39 2022] POST /identity/v3/projects => generated 363 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:39.580783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:39.596072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fd044-8c95-4734-bdf5-86925b02ad14 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:39.664539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475df398-16f7-4ddc-bfac-83bfb38c7fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:39.665407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df398-16f7-4ddc-bfac-83bfb38c7fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.665802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df398-16f7-4ddc-bfac-83bfb38c7fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.666177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475df398-16f7-4ddc-bfac-83bfb38c7fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.666995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475df398-16f7-4ddc-bfac-83bfb38c7fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:39.669771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475df398-16f7-4ddc-bfac-83bfb38c7fc8 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:39.815951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1377/2765] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:39 2022] POST /identity/v3/users => generated 282 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:39.820512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f316e2-af80-49ad-920d-1c20f4103630 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:39.834867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f316e2-af80-49ad-920d-1c20f4103630 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:39.841496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:39.842355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.842742 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.843121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.844001 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:39.845779 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:39.863827 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475df398-16f7-4ddc-bfac-83bfb38c7fc8 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:39.869356 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1389/2766] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:39.875033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475df398-16f7-4ddc-bfac-83bfb38c7fc8 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:39.891346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04ebe2-117f-4d86-a8b6-718c2feb3cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:39.892000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04ebe2-117f-4d86-a8b6-718c2feb3cad admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.892576 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04ebe2-117f-4d86-a8b6-718c2feb3cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.893048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04ebe2-117f-4d86-a8b6-718c2feb3cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.894123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04ebe2-117f-4d86-a8b6-718c2feb3cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:39.900921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04ebe2-117f-4d86-a8b6-718c2feb3cad admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:39.901806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:39.905147 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:39.920252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1378/2767] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:39.928833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04ebe2-117f-4d86-a8b6-718c2feb3cad admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:39.940540 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5e8aa-c3ca-4ac1-ab7a-5485d9582f6c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:39.944280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04ebe2-117f-4d86-a8b6-718c2feb3cad admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:39.949367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:39.950590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.951045 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.951524 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] PATH_INFO: `/v3/users/9b30ca943dd845cbb45c8d302f35777f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.951909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36577898-3b16-4699-a1be-ebb7042e641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:39.953460 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36577898-3b16-4699-a1be-ebb7042e641a admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:39.960120 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36577898-3b16-4699-a1be-ebb7042e641a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:39.960265 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b30ca943dd845cbb45c8d302f35777f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:39.960423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36577898-3b16-4699-a1be-ebb7042e641a admin admin] PATH_INFO: `/v3/projects/e07f2224fa5f4428a483d8ba9c1c8b56/users/1a665969398b4100aababcc7da106541/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:39.960514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:39.982921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36577898-3b16-4699-a1be-ebb7042e641a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e07f2224fa5f4428a483d8ba9c1c8b56, user_id=1a665969398b4100aababcc7da106541, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:39.986158 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36577898-3b16-4699-a1be-ebb7042e641a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:40.113415 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1379/2768] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:39 2022] PUT /identity/v3/projects/e07f2224fa5f4428a483d8ba9c1c8b56/users/1a665969398b4100aababcc7da106541/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:40.124208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602384f-56fa-406c-85d1-7c0e252ccd86 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.124506 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602384f-56fa-406c-85d1-7c0e252ccd86 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.124788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602384f-56fa-406c-85d1-7c0e252ccd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.157190 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5602384f-56fa-406c-85d1-7c0e252ccd86 None None] MFA Rules not processed for user `1a665969398b4100aababcc7da106541`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:40.161538 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5602384f-56fa-406c-85d1-7c0e252ccd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:40.189517 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:40.237953 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1380/2769] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:40.239064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602384f-56fa-406c-85d1-7c0e252ccd86 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:40.275221 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b30ca943dd845cbb45c8d302f35777f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:40.276085 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b30ca943dd845cbb45c8d302f35777f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:40.276788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b30ca943dd845cbb45c8d302f35777f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:40.290213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b30ca943dd845cbb45c8d302f35777f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:40.305951 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-803e62c9-5057-46ab-94ac-5be2adb93119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b30ca943dd845cbb45c8d302f35777f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:40.317243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602384f-56fa-406c-85d1-7c0e252ccd86 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:40.326913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:40.327761 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.328086 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.328395 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.329277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:40.331358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:40.391352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1390/2770] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:39 2022] DELETE /identity/v3/users/9b30ca943dd845cbb45c8d302f35777f => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:40.393206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268f06-56bd-4092-adaf-1043272d4b3b None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.393522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268f06-56bd-4092-adaf-1043272d4b3b None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.393805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268f06-56bd-4092-adaf-1043272d4b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.413302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6e268f06-56bd-4092-adaf-1043272d4b3b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:40.414009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6e268f06-56bd-4092-adaf-1043272d4b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:40.440760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:40.477470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1391/2771] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:40.490210 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e268f06-56bd-4092-adaf-1043272d4b3b None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:40.492412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6e268f06-56bd-4092-adaf-1043272d4b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:40.550601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e268f06-56bd-4092-adaf-1043272d4b3b None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:40.555608 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1381/2772] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:40 2022] POST /identity/v3/projects => generated 365 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:40.556898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:40.559896 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:40.624598 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6639536-9291-4978-9699-5ac06cff51d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:40.625276 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639536-9291-4978-9699-5ac06cff51d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.625649 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639536-9291-4978-9699-5ac06cff51d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.625934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6639536-9291-4978-9699-5ac06cff51d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.671955 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4c522-d7e3-4426-b3a9-53571b145218 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:40.677268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540704d-e5ed-4238-ae15-15117d001f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:40.678252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540704d-e5ed-4238-ae15-15117d001f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.678570 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540704d-e5ed-4238-ae15-15117d001f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.678924 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540704d-e5ed-4238-ae15-15117d001f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.679732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540704d-e5ed-4238-ae15-15117d001f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:40.682180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540704d-e5ed-4238-ae15-15117d001f27 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:40.714732 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6639536-9291-4978-9699-5ac06cff51d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:40.717180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6639536-9291-4978-9699-5ac06cff51d0 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:40.728662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1392/2773] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:40.789884 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1382/2774] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:40 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 12:00:40.799124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6639536-9291-4978-9699-5ac06cff51d0 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:40.809208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6639536-9291-4978-9699-5ac06cff51d0 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:40.813934 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6683e-a171-4740-846c-7bfbcba0b533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:40.814653 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6683e-a171-4740-846c-7bfbcba0b533 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.816573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6683e-a171-4740-846c-7bfbcba0b533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.816573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6683e-a171-4740-846c-7bfbcba0b533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.816573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6683e-a171-4740-846c-7bfbcba0b533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:40.817635 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6683e-a171-4740-846c-7bfbcba0b533 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:40.829497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1393/2775] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:40.833683 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540704d-e5ed-4238-ae15-15117d001f27 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:40.844441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540704d-e5ed-4238-ae15-15117d001f27 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:40.849156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfd08a-6479-4775-9c12-75dcd6356ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:40.850052 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfd08a-6479-4775-9c12-75dcd6356ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:40.850333 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfd08a-6479-4775-9c12-75dcd6356ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:40.850662 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfd08a-6479-4775-9c12-75dcd6356ccd admin admin] PATH_INFO: `/v3/projects/6e4403413f9643899ad588293992d779/users/bf24e1d9707b4ff498aa50c450a6e263/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:40.868801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfd08a-6479-4775-9c12-75dcd6356ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4403413f9643899ad588293992d779, user_id=bf24e1d9707b4ff498aa50c450a6e263, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:40.871589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfd08a-6479-4775-9c12-75dcd6356ccd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:40.976626 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1383/2776] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:40 2022] PUT /identity/v3/projects/6e4403413f9643899ad588293992d779/users/bf24e1d9707b4ff498aa50c450a6e263/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:40.981586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6683e-a171-4740-846c-7bfbcba0b533 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:40.993957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6683e-a171-4740-846c-7bfbcba0b533 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:41.061286 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:41.062073 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:41.062328 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:41.062605 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:41.063456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:41.065049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:41.084586 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1394/2777] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:41.085840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfd08a-6479-4775-9c12-75dcd6356ccd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:41.097369 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfd08a-6479-4775-9c12-75dcd6356ccd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:41.102196 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c241afe-9511-478c-8eb1-e142f660b031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:41.103364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241afe-9511-478c-8eb1-e142f660b031 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:41.106723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241afe-9511-478c-8eb1-e142f660b031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:41.106723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c241afe-9511-478c-8eb1-e142f660b031 admin admin] PATH_INFO: `/v3/projects/6e4403413f9643899ad588293992d779/users/bf24e1d9707b4ff498aa50c450a6e263/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:41.107407 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c241afe-9511-478c-8eb1-e142f660b031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4403413f9643899ad588293992d779, user_id=bf24e1d9707b4ff498aa50c450a6e263, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:41.110095 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c241afe-9511-478c-8eb1-e142f660b031 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:41.191639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:41.193577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:41.199817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=100 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:41.209291 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d21114-539f-4379-93dd-19e037714ac2 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:41.220290 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:41.224744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:41.224744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:41.224744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] PATH_INFO: `/v3/projects/2c165a933868449fa4872dad9b3cb1c2` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:41.224744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c165a933868449fa4872dad9b3cb1c2)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:41.225362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:41.227897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1384/2778] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:41 2022] PUT /identity/v3/projects/6e4403413f9643899ad588293992d779/users/bf24e1d9707b4ff498aa50c450a6e263/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:41.238360 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73066bb-9a95-475c-9d8e-8799941a7f36 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:41.238710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73066bb-9a95-475c-9d8e-8799941a7f36 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:41.238996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73066bb-9a95-475c-9d8e-8799941a7f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:41.246817 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:41.269802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-b73066bb-9a95-475c-9d8e-8799941a7f36 None None] MFA Rules not processed for user `bf24e1d9707b4ff498aa50c450a6e263`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:41.271044 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b73066bb-9a95-475c-9d8e-8799941a7f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:41.338094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c165a933868449fa4872dad9b3cb1c2', 'request_id': 'req-840a3433-2059-4b99-a995-c4177dff3cdd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.342056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1385/2779] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:41.344382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c165a933868449fa4872dad9b3cb1c2', 'request_id': 'req-840a3433-2059-4b99-a995-c4177dff3cdd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.357408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73066bb-9a95-475c-9d8e-8799941a7f36 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:41.358923 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-b73066bb-9a95-475c-9d8e-8799941a7f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:41.403912 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c165a933868449fa4872dad9b3cb1c2', 'request_id': 'req-840a3433-2059-4b99-a995-c4177dff3cdd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.417125 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] The token cache is being invalidate because project 2c165a933868449fa4872dad9b3cb1c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:41.417512 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.418306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.419568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73066bb-9a95-475c-9d8e-8799941a7f36 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:41.420029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1395/2780] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:41 2022] DELETE /identity/v3/projects/2c165a933868449fa4872dad9b3cb1c2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:41.423816 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:41.425208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:41.469375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:41.469932 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:41.470218 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:41.470525 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:41.472338 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:41.497304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a3433-2059-4b99-a995-c4177dff3cdd admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:41.501914 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:41.502786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:41.503070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:41.503375 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] PATH_INFO: `/v3/users/724e64eb334e4dc1a3b24ab0fd8836cd` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:41.504569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724e64eb334e4dc1a3b24ab0fd8836cd)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:41.508352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:41.537718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:41.541364 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:41.542842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:41.552590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1386/2781] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:41.694493 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:41.696725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724e64eb334e4dc1a3b24ab0fd8836cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.697437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724e64eb334e4dc1a3b24ab0fd8836cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.698287 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724e64eb334e4dc1a3b24ab0fd8836cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.705614 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724e64eb334e4dc1a3b24ab0fd8836cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.717015 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724e64eb334e4dc1a3b24ab0fd8836cd'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:41.791940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:41.793852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:41.810539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ff1a7-6e2b-4c36-89d5-2cd6d581ce52 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:41.811692 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1396/2782] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:41 2022] DELETE /identity/v3/users/724e64eb334e4dc1a3b24ab0fd8836cd => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:41.884194 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02674676-4ef8-4cd9-9084-402215c09548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:41.884988 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674676-4ef8-4cd9-9084-402215c09548 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:41.885358 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674676-4ef8-4cd9-9084-402215c09548 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:41.885739 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02674676-4ef8-4cd9-9084-402215c09548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:41.889531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-02674676-4ef8-4cd9-9084-402215c09548 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:41.960336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02674676-4ef8-4cd9-9084-402215c09548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:41.961784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02674676-4ef8-4cd9-9084-402215c09548 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:41.975072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1387/2783] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:41 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:42.331806 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:42.333253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:42.406202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dbe65-0169-4848-b248-74fe1c1c2cb6 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:42.411538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:42.412455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:42.412748 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:42.413080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] PATH_INFO: `/v3/projects/4e0fc9ee0dcf4b9e8f1df879108f913f` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:42.415773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0fc9ee0dcf4b9e8f1df879108f913f)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:42.418161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:42.566979 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0fc9ee0dcf4b9e8f1df879108f913f', 'request_id': 'req-a6dd4707-d039-448f-b7c9-62be3c20a13f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:42.573783 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0fc9ee0dcf4b9e8f1df879108f913f', 'request_id': 'req-a6dd4707-d039-448f-b7c9-62be3c20a13f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:42.659729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0fc9ee0dcf4b9e8f1df879108f913f', 'request_id': 'req-a6dd4707-d039-448f-b7c9-62be3c20a13f'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:42.671499 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] The token cache is being invalidate because project 4e0fc9ee0dcf4b9e8f1df879108f913f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:42.671800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:42.672462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-a6dd4707-d039-448f-b7c9-62be3c20a13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:42.674020 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1397/2784] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:42 2022] DELETE /identity/v3/projects/4e0fc9ee0dcf4b9e8f1df879108f913f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:42.679014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbdca-f970-443c-91d8-51e9dc4e8cbf None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:42.679547 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbdca-f970-443c-91d8-51e9dc4e8cbf None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:42.679637 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbdca-f970-443c-91d8-51e9dc4e8cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:42.698723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4c7dbdca-f970-443c-91d8-51e9dc4e8cbf None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:42.700578 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4c7dbdca-f970-443c-91d8-51e9dc4e8cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:42.769393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1388/2785] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:42.778267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdc469-a707-4daa-8f9c-242b0a810542 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:42.778561 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdc469-a707-4daa-8f9c-242b0a810542 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:42.778850 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdc469-a707-4daa-8f9c-242b0a810542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:42.797162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-16cdc469-a707-4daa-8f9c-242b0a810542 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:42.798366 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-16cdc469-a707-4daa-8f9c-242b0a810542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:42.807196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1398/2786] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:42.812059 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dbdca-f970-443c-91d8-51e9dc4e8cbf None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:42.821909 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dbdca-f970-443c-91d8-51e9dc4e8cbf None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:42.827072 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:42.827692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:42.827975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:42.828250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:42.828965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:00:42.829389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:42.831240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:42.843379 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1389/2787] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:00:42.847794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdc469-a707-4daa-8f9c-242b0a810542 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:42.857153 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdc469-a707-4daa-8f9c-242b0a810542 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:42.862750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357018f0-5ccb-4b64-ae65-7fe4e6c91051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:42.863485 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357018f0-5ccb-4b64-ae65-7fe4e6c91051 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:42.863918 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357018f0-5ccb-4b64-ae65-7fe4e6c91051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:42.864206 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357018f0-5ccb-4b64-ae65-7fe4e6c91051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:42.864919 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357018f0-5ccb-4b64-ae65-7fe4e6c91051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:42.866855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357018f0-5ccb-4b64-ae65-7fe4e6c91051 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:42.958990 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1399/2788] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:42 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:42.963136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:42.975487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3be989-46f0-4cde-a5bd-20daa27d2ee2 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:43.010619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:43.040530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf5e2d-2b2e-47e0-a20e-7b2ba0e6828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:43.041206 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf5e2d-2b2e-47e0-a20e-7b2ba0e6828b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:43.041660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf5e2d-2b2e-47e0-a20e-7b2ba0e6828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:43.042308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf5e2d-2b2e-47e0-a20e-7b2ba0e6828b admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:43.045397 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf5e2d-2b2e-47e0-a20e-7b2ba0e6828b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:43.049121 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf5e2d-2b2e-47e0-a20e-7b2ba0e6828b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:43.164559 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1390/2789] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:42 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:43.170456 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357018f0-5ccb-4b64-ae65-7fe4e6c91051 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:43.181185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357018f0-5ccb-4b64-ae65-7fe4e6c91051 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:43.187484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ddcba-979a-4849-9dd3-e56804dc7c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:43.188380 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ddcba-979a-4849-9dd3-e56804dc7c18 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:43.188773 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ddcba-979a-4849-9dd3-e56804dc7c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:43.189057 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ddcba-979a-4849-9dd3-e56804dc7c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:43.189805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ddcba-979a-4849-9dd3-e56804dc7c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:43.191611 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ddcba-979a-4849-9dd3-e56804dc7c18 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:43.204639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1400/2790] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:43.209419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf5e2d-2b2e-47e0-a20e-7b2ba0e6828b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:43.221704 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf5e2d-2b2e-47e0-a20e-7b2ba0e6828b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:43.226298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337eec4-4409-4f2b-af46-293e3ad9c1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:43.226960 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337eec4-4409-4f2b-af46-293e3ad9c1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:43.227242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337eec4-4409-4f2b-af46-293e3ad9c1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:43.228238 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337eec4-4409-4f2b-af46-293e3ad9c1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:43.228372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337eec4-4409-4f2b-af46-293e3ad9c1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:43.229944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337eec4-4409-4f2b-af46-293e3ad9c1f3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:43.242117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1391/2791] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:43.247142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ddcba-979a-4849-9dd3-e56804dc7c18 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:43.257252 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ddcba-979a-4849-9dd3-e56804dc7c18 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:43.261986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:43.263051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:43.263388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:43.263676 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] PATH_INFO: `/v3/projects/e2d0c9baa584498494257302e9d5d98d/users/06324752e1c043c2a4f34d3751efbcfb/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:43.275300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:43.279215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d0c9baa584498494257302e9d5d98d, user_id=06324752e1c043c2a4f34d3751efbcfb, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:43.281945 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:43.386076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1401/2792] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:43 2022] PUT /identity/v3/projects/e2d0c9baa584498494257302e9d5d98d/users/06324752e1c043c2a4f34d3751efbcfb/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:43.396239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8621-3436-45d2-9222-a10993cbd8d0 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:43.396531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8621-3436-45d2-9222-a10993cbd8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:43.396810 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8621-3436-45d2-9222-a10993cbd8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:43.415423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-62eb8621-3436-45d2-9222-a10993cbd8d0 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:43.416667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-62eb8621-3436-45d2-9222-a10993cbd8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:43.478047 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1392/2793] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:43.488622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:43.490465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:43.549758 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1237e-3346-41de-935d-0e70a5e670ec admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:43.553384 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33e397-449f-4f16-a760-163883b7f29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:43.554035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33e397-449f-4f16-a760-163883b7f29f service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:43.554316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33e397-449f-4f16-a760-163883b7f29f service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:43.554590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33e397-449f-4f16-a760-163883b7f29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:43.563762 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33e397-449f-4f16-a760-163883b7f29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:43.565184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33e397-449f-4f16-a760-163883b7f29f service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:43.576683 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1402/2794] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:45.006103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d98865-ca1f-44ae-bc01-b6116ae0ca85 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:45.006660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d98865-ca1f-44ae-bc01-b6116ae0ca85 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:45.006660 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d98865-ca1f-44ae-bc01-b6116ae0ca85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:45.037600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-80d98865-ca1f-44ae-bc01-b6116ae0ca85 None None] MFA Rules not processed for user `06324752e1c043c2a4f34d3751efbcfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:45.038978 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-80d98865-ca1f-44ae-bc01-b6116ae0ca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:45.122522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1393/2795] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:45.127109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33e397-449f-4f16-a760-163883b7f29f service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:45.137409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33e397-449f-4f16-a760-163883b7f29f service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:45.143855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6b8de-27cf-4abc-a879-c829c2ec587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:45.144538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b8de-27cf-4abc-a879-c829c2ec587c admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:45.144855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b8de-27cf-4abc-a879-c829c2ec587c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:45.145136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6b8de-27cf-4abc-a879-c829c2ec587c admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:45.145844 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6b8de-27cf-4abc-a879-c829c2ec587c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:45.147788 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6b8de-27cf-4abc-a879-c829c2ec587c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:45.279032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1403/2796] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:45 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:45.283344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d98865-ca1f-44ae-bc01-b6116ae0ca85 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:45.295017 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d98865-ca1f-44ae-bc01-b6116ae0ca85 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:45.338085 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:45.353004 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afce31a-616a-4610-800c-0137d506bd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:45.353684 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afce31a-616a-4610-800c-0137d506bd6d admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:45.353968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afce31a-616a-4610-800c-0137d506bd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:45.354245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afce31a-616a-4610-800c-0137d506bd6d admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:45.354943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afce31a-616a-4610-800c-0137d506bd6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:45.356921 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afce31a-616a-4610-800c-0137d506bd6d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:45.440856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1394/2797] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:45 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:45.442309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6b8de-27cf-4abc-a879-c829c2ec587c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:45.454687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6b8de-27cf-4abc-a879-c829c2ec587c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:45.459490 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d7a73-c159-49f5-89ea-09ad6353fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:45.460300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d7a73-c159-49f5-89ea-09ad6353fbad admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:45.460595 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d7a73-c159-49f5-89ea-09ad6353fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:45.460873 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d7a73-c159-49f5-89ea-09ad6353fbad admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:45.461596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d7a73-c159-49f5-89ea-09ad6353fbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:45.463185 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1d7a73-c159-49f5-89ea-09ad6353fbad admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:45.475367 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1404/2798] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:45.479676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afce31a-616a-4610-800c-0137d506bd6d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:45.489769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afce31a-616a-4610-800c-0137d506bd6d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:45.494868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:45.496088 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:45.496462 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:45.496831 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] PATH_INFO: `/v3/projects/d5983a7bc2fe4427b7bc24ea41c65eda/users/189c710a05d34f2e8e87069c7377e6cf/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:45.518906 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5983a7bc2fe4427b7bc24ea41c65eda, user_id=189c710a05d34f2e8e87069c7377e6cf, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:45.524242 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:45.607743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1395/2799] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:45 2022] PUT /identity/v3/projects/d5983a7bc2fe4427b7bc24ea41c65eda/users/189c710a05d34f2e8e87069c7377e6cf/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:45.611009 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d7a73-c159-49f5-89ea-09ad6353fbad admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:45.621089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d7a73-c159-49f5-89ea-09ad6353fbad admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:45.647420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:45.649589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:45.682813 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458022-e2cd-4871-87f5-9c34f0529a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:45.684046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458022-e2cd-4871-87f5-9c34f0529a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:45.684332 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458022-e2cd-4871-87f5-9c34f0529a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:45.684627 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93458022-e2cd-4871-87f5-9c34f0529a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:45.685370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458022-e2cd-4871-87f5-9c34f0529a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:45.686999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93458022-e2cd-4871-87f5-9c34f0529a01 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:45.697642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1405/2800] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:45.702238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458022-e2cd-4871-87f5-9c34f0529a01 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:45.712196 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93458022-e2cd-4871-87f5-9c34f0529a01 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:45.721046 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73911a53-0a60-4c8d-9543-1668b0bb6563 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:45.722217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1a6c9-e7e4-432b-89b3-152c6dd9e8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:45.723197 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1a6c9-e7e4-432b-89b3-152c6dd9e8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:45.726032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1a6c9-e7e4-432b-89b3-152c6dd9e8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:45.726032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1a6c9-e7e4-432b-89b3-152c6dd9e8a9 admin admin] PATH_INFO: `/v3/projects/d5983a7bc2fe4427b7bc24ea41c65eda/users/189c710a05d34f2e8e87069c7377e6cf/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:45.726211 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:45.726308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1a6c9-e7e4-432b-89b3-152c6dd9e8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5983a7bc2fe4427b7bc24ea41c65eda, user_id=189c710a05d34f2e8e87069c7377e6cf, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:45.726706 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:45.726990 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:45.727298 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] PATH_INFO: `/v3/users/1a665969398b4100aababcc7da106541` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:45.728502 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a665969398b4100aababcc7da106541)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:45.729111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff1a6c9-e7e4-432b-89b3-152c6dd9e8a9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:45.730486 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:45.740168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1406/2801] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:45 2022] PUT /identity/v3/projects/d5983a7bc2fe4427b7bc24ea41c65eda/users/189c710a05d34f2e8e87069c7377e6cf/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:45.907993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a665969398b4100aababcc7da106541'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:45.908664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a665969398b4100aababcc7da106541'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:45.909281 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a665969398b4100aababcc7da106541'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:45.916331 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a665969398b4100aababcc7da106541'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:45.930344 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a665969398b4100aababcc7da106541'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:46.035871 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1396/2802] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:45 2022] DELETE /identity/v3/users/1a665969398b4100aababcc7da106541 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:46.040029 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db918f-e8ac-474f-9b70-86403f5286c5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:46.040480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db918f-e8ac-474f-9b70-86403f5286c5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:46.040890 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db918f-e8ac-474f-9b70-86403f5286c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:46.063025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e7db918f-e8ac-474f-9b70-86403f5286c5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:46.064269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e7db918f-e8ac-474f-9b70-86403f5286c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:46.119133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:46.141313 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1407/2803] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:46.153941 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:46.155504 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:46.188080 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:46.215730 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318ed59-aa65-4c17-9252-bea09848f5e5 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:46.219630 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaa94d-0bd7-4689-be99-82e7b915a3a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:46.220394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa94d-0bd7-4689-be99-82e7b915a3a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:46.220686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa94d-0bd7-4689-be99-82e7b915a3a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:46.220965 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa94d-0bd7-4689-be99-82e7b915a3a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:46.230554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaa94d-0bd7-4689-be99-82e7b915a3a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:46.232034 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaa94d-0bd7-4689-be99-82e7b915a3a1 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:46.243012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1397/2804] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:46.795794 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db918f-e8ac-474f-9b70-86403f5286c5 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:46.797474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e7db918f-e8ac-474f-9b70-86403f5286c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:46.812162 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db918f-e8ac-474f-9b70-86403f5286c5 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:46.816775 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:46.817745 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:46.818022 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:46.818302 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] PATH_INFO: `/v3/projects/e07f2224fa5f4428a483d8ba9c1c8b56` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:46.821569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e07f2224fa5f4428a483d8ba9c1c8b56)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:46.821667 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:46.955841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e07f2224fa5f4428a483d8ba9c1c8b56', 'request_id': 'req-dd9db9c0-188c-457b-956c-6b9016e6e915'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:46.961771 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e07f2224fa5f4428a483d8ba9c1c8b56', 'request_id': 'req-dd9db9c0-188c-457b-956c-6b9016e6e915'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.046409 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e07f2224fa5f4428a483d8ba9c1c8b56', 'request_id': 'req-dd9db9c0-188c-457b-956c-6b9016e6e915'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.062892 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] The token cache is being invalidate because project e07f2224fa5f4428a483d8ba9c1c8b56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:47.063191 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.063910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-dd9db9c0-188c-457b-956c-6b9016e6e915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.065639 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1408/2805] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:46 2022] DELETE /identity/v3/projects/e07f2224fa5f4428a483d8ba9c1c8b56 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:47.070485 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaa94d-0bd7-4689-be99-82e7b915a3a1 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:47.086530 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaa94d-0bd7-4689-be99-82e7b915a3a1 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:47.160102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:47.160993 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:47.161283 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:47.161561 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] PATH_INFO: `/v3/users/bf24e1d9707b4ff498aa50c450a6e263` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:47.162779 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf24e1d9707b4ff498aa50c450a6e263)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:47.162883 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab74c-0aa7-42ad-bd0e-b6a161ebe986 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:47.163328 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab74c-0aa7-42ad-bd0e-b6a161ebe986 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:47.163538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab74c-0aa7-42ad-bd0e-b6a161ebe986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:47.164931 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:47.198135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f8dab74c-0aa7-42ad-bd0e-b6a161ebe986 None None] MFA Rules not processed for user `189c710a05d34f2e8e87069c7377e6cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:47.199446 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f8dab74c-0aa7-42ad-bd0e-b6a161ebe986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:47.289173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1409/2806] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:47.301208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab74c-0aa7-42ad-bd0e-b6a161ebe986 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:47.302527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f8dab74c-0aa7-42ad-bd0e-b6a161ebe986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:47.362664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab74c-0aa7-42ad-bd0e-b6a161ebe986 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:47.368089 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:47.368894 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:47.369180 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:47.369462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:47.371457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:47.377935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:47.396524 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf24e1d9707b4ff498aa50c450a6e263'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.397176 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf24e1d9707b4ff498aa50c450a6e263'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.397854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf24e1d9707b4ff498aa50c450a6e263'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.406892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf24e1d9707b4ff498aa50c450a6e263'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.420145 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-409e215d-1e3c-429d-a679-68cd19f9b56f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf24e1d9707b4ff498aa50c450a6e263'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:47.434908 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:47.436494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:47.445474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1410/2807] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:47.514766 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1398/2808] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:47 2022] DELETE /identity/v3/users/bf24e1d9707b4ff498aa50c450a6e263 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:47.936736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:47.938465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:47.956707 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ceba0-a275-40ee-b304-70c71bbef673 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:48.013649 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:48.014510 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.014795 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.015074 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] PATH_INFO: `/v3/projects/6e4403413f9643899ad588293992d779` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.016638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4403413f9643899ad588293992d779)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:48.018532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:48.149943 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4403413f9643899ad588293992d779', 'request_id': 'req-80fc1ed1-a700-45e6-9120-46b147b55a67'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:48.158695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4403413f9643899ad588293992d779', 'request_id': 'req-80fc1ed1-a700-45e6-9120-46b147b55a67'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:48.268915 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4403413f9643899ad588293992d779', 'request_id': 'req-80fc1ed1-a700-45e6-9120-46b147b55a67'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:48.280633 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] The token cache is being invalidate because project 6e4403413f9643899ad588293992d779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:48.280925 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:48.281546 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-80fc1ed1-a700-45e6-9120-46b147b55a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:48.283078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1411/2809] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:47 2022] DELETE /identity/v3/projects/6e4403413f9643899ad588293992d779 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:48.288180 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7d265-d3c6-471f-b4db-f71e66a99972 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.288665 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7d265-d3c6-471f-b4db-f71e66a99972 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.288778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7d265-d3c6-471f-b4db-f71e66a99972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.306026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-51f7d265-d3c6-471f-b4db-f71e66a99972 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:48.307213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-51f7d265-d3c6-471f-b4db-f71e66a99972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:48.352208 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:48.367079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1399/2810] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:48.377955 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f93e4-b918-48df-a210-787da8daabbc None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.383203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f93e4-b918-48df-a210-787da8daabbc None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.383203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f93e4-b918-48df-a210-787da8daabbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.397227 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-ee7f93e4-b918-48df-a210-787da8daabbc None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:48.398475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ee7f93e4-b918-48df-a210-787da8daabbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:48.406440 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1412/2811] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:48.414457 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7d265-d3c6-471f-b4db-f71e66a99972 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:48.425472 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7d265-d3c6-471f-b4db-f71e66a99972 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:48.430494 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:48.431148 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.431459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.431736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.432454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:00:48.432757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:48.435014 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:48.447223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1400/2812] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:00:48.451566 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f93e4-b918-48df-a210-787da8daabbc None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:48.460942 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f93e4-b918-48df-a210-787da8daabbc None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:48.465778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa61e20-82ff-4a39-b65f-18d4b35f5cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:48.466457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61e20-82ff-4a39-b65f-18d4b35f5cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.466736 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61e20-82ff-4a39-b65f-18d4b35f5cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.467016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa61e20-82ff-4a39-b65f-18d4b35f5cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.467830 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa61e20-82ff-4a39-b65f-18d4b35f5cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:48.469778 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa61e20-82ff-4a39-b65f-18d4b35f5cd7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:48.554477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1413/2813] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:48 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:48.556441 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:48.567191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468fae5-fd55-45c1-a756-033976aa74bd admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:48.623179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b12867-f663-4c41-b037-e945cddbb5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:48.623930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12867-f663-4c41-b037-e945cddbb5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.624217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12867-f663-4c41-b037-e945cddbb5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.624805 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12867-f663-4c41-b037-e945cddbb5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.625200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b12867-f663-4c41-b037-e945cddbb5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:48.627151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b12867-f663-4c41-b037-e945cddbb5d9 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:48.725777 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1401/2814] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:48 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:48.727061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa61e20-82ff-4a39-b65f-18d4b35f5cd7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:48.738130 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa61e20-82ff-4a39-b65f-18d4b35f5cd7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:48.754002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248b751-95e3-48b3-a507-1002fb7f23ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:48.754002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248b751-95e3-48b3-a507-1002fb7f23ee admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.754002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248b751-95e3-48b3-a507-1002fb7f23ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.754002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248b751-95e3-48b3-a507-1002fb7f23ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.754369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248b751-95e3-48b3-a507-1002fb7f23ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:48.754369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248b751-95e3-48b3-a507-1002fb7f23ee admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:48.754369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:48.764437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1414/2815] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 12:00:48.765678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b12867-f663-4c41-b037-e945cddbb5d9 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:48.778248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b12867-f663-4c41-b037-e945cddbb5d9 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:48.783377 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e8901-0786-4c38-a054-61c959c6650d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:48.783898 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e8901-0786-4c38-a054-61c959c6650d admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.784182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e8901-0786-4c38-a054-61c959c6650d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.784454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e8901-0786-4c38-a054-61c959c6650d admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.785186 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e8901-0786-4c38-a054-61c959c6650d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:48.786792 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e8901-0786-4c38-a054-61c959c6650d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:48.798350 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1402/2816] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:48.802518 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248b751-95e3-48b3-a507-1002fb7f23ee admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:48.812731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248b751-95e3-48b3-a507-1002fb7f23ee admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:48.817340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c75d3c-d67f-4ba0-ba95-a8f57a06d21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:48.818240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c75d3c-d67f-4ba0-ba95-a8f57a06d21c admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.818557 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c75d3c-d67f-4ba0-ba95-a8f57a06d21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.818798 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c75d3c-d67f-4ba0-ba95-a8f57a06d21c admin admin] PATH_INFO: `/v3/projects/da7fcd45e9224a1688fe535254887bdc/users/a04856742a0b4c179d37aa69ddf45c55/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.834386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c75d3c-d67f-4ba0-ba95-a8f57a06d21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7fcd45e9224a1688fe535254887bdc, user_id=a04856742a0b4c179d37aa69ddf45c55, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:48.837148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c75d3c-d67f-4ba0-ba95-a8f57a06d21c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:48.931814 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1415/2817] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:48 2022] PUT /identity/v3/projects/da7fcd45e9224a1688fe535254887bdc/users/a04856742a0b4c179d37aa69ddf45c55/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:48.942927 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00778ff-d167-4469-9ff0-db0cad9a7332 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:48.943322 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00778ff-d167-4469-9ff0-db0cad9a7332 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:48.943671 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00778ff-d167-4469-9ff0-db0cad9a7332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:48.975480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c00778ff-d167-4469-9ff0-db0cad9a7332 None None] MFA Rules not processed for user `a04856742a0b4c179d37aa69ddf45c55`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:48.977984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c00778ff-d167-4469-9ff0-db0cad9a7332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:49.047869 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1403/2818] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:49.052723 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c75d3c-d67f-4ba0-ba95-a8f57a06d21c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:49.121309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c75d3c-d67f-4ba0-ba95-a8f57a06d21c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:49.126295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:49.126994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.127304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.127589 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.128283 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:49.130379 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:49.215294 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1416/2819] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:49 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:49.219280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00778ff-d167-4469-9ff0-db0cad9a7332 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:49.224369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:49.226107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:49.233084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00778ff-d167-4469-9ff0-db0cad9a7332 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:49.279005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:49.290774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d206f6-6550-449f-88ec-8136631a5a35 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:49.294530 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:49.295188 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.295509 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.295792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.297601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:49.298592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f285d79-1aeb-425f-8a29-7d588364fc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:49.299422 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f285d79-1aeb-425f-8a29-7d588364fc3d admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.299716 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f285d79-1aeb-425f-8a29-7d588364fc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.299995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f285d79-1aeb-425f-8a29-7d588364fc3d admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.301246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f285d79-1aeb-425f-8a29-7d588364fc3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:49.303371 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f285d79-1aeb-425f-8a29-7d588364fc3d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:49.363139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:49.364643 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:49.374232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1417/2820] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:49.407191 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1404/2821] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:49 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:49.410902 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:49.421241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47f7af-6e1b-42d6-86f4-4cff81d8aded service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:49.426267 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d2436-88ef-433d-8898-bab34d5a1b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:49.426950 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2436-88ef-433d-8898-bab34d5a1b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.427237 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2436-88ef-433d-8898-bab34d5a1b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.427539 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2436-88ef-433d-8898-bab34d5a1b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.428259 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d2436-88ef-433d-8898-bab34d5a1b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:49.429821 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d2436-88ef-433d-8898-bab34d5a1b4b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:49.431155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:49.441121 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1418/2822] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:49.445166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f285d79-1aeb-425f-8a29-7d588364fc3d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:49.456079 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f285d79-1aeb-425f-8a29-7d588364fc3d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:49.460765 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9ecea-abaf-4cfd-af3c-a977d46bbc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:49.461718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9ecea-abaf-4cfd-af3c-a977d46bbc72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.461945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9ecea-abaf-4cfd-af3c-a977d46bbc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.462224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9ecea-abaf-4cfd-af3c-a977d46bbc72 admin admin] PATH_INFO: `/v3/projects/9409b73f16404eab9dae8cfdef042fd6/users/01e2840516964ddd9fa9594c82d76bf6/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.477009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9ecea-abaf-4cfd-af3c-a977d46bbc72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9409b73f16404eab9dae8cfdef042fd6, user_id=01e2840516964ddd9fa9594c82d76bf6, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:49.479870 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9ecea-abaf-4cfd-af3c-a977d46bbc72 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:49.574943 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1405/2823] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:49 2022] PUT /identity/v3/projects/9409b73f16404eab9dae8cfdef042fd6/users/01e2840516964ddd9fa9594c82d76bf6/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:49.579163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d2436-88ef-433d-8898-bab34d5a1b4b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:49.589985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d2436-88ef-433d-8898-bab34d5a1b4b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:49.646062 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fe356-0099-4010-bfb5-31149461c379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:49.646760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fe356-0099-4010-bfb5-31149461c379 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.647034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fe356-0099-4010-bfb5-31149461c379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.647340 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fe356-0099-4010-bfb5-31149461c379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.648058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fe356-0099-4010-bfb5-31149461c379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:49.649636 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fe356-0099-4010-bfb5-31149461c379 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:49.659163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1419/2824] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:49.663542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9ecea-abaf-4cfd-af3c-a977d46bbc72 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:49.673833 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9ecea-abaf-4cfd-af3c-a977d46bbc72 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:49.678266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d2fe2-940e-4cb7-af1f-bb68e24aa9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:49.679164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d2fe2-940e-4cb7-af1f-bb68e24aa9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.679475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d2fe2-940e-4cb7-af1f-bb68e24aa9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.696245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d2fe2-940e-4cb7-af1f-bb68e24aa9d0 admin admin] PATH_INFO: `/v3/projects/9409b73f16404eab9dae8cfdef042fd6/users/01e2840516964ddd9fa9594c82d76bf6/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.696245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d2fe2-940e-4cb7-af1f-bb68e24aa9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9409b73f16404eab9dae8cfdef042fd6, user_id=01e2840516964ddd9fa9594c82d76bf6, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:49.696245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d2fe2-940e-4cb7-af1f-bb68e24aa9d0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:49.696245 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1406/2825] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:49 2022] PUT /identity/v3/projects/9409b73f16404eab9dae8cfdef042fd6/users/01e2840516964ddd9fa9594c82d76bf6/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:49.705657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312aa3-65fe-433c-a6f1-1d6b4e34e366 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.706008 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312aa3-65fe-433c-a6f1-1d6b4e34e366 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.706399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312aa3-65fe-433c-a6f1-1d6b4e34e366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.732766 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-db312aa3-65fe-433c-a6f1-1d6b4e34e366 None None] MFA Rules not processed for user `01e2840516964ddd9fa9594c82d76bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:49.733978 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-db312aa3-65fe-433c-a6f1-1d6b4e34e366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:49.806426 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1420/2826] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:49.830595 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d2fe2-940e-4cb7-af1f-bb68e24aa9d0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:49.890005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d2fe2-940e-4cb7-af1f-bb68e24aa9d0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:49.893799 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d40d6-07ad-43b2-b93a-96a81116d503 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:49.894444 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d40d6-07ad-43b2-b93a-96a81116d503 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:49.894725 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d40d6-07ad-43b2-b93a-96a81116d503 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:49.895002 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d40d6-07ad-43b2-b93a-96a81116d503 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:49.904056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d40d6-07ad-43b2-b93a-96a81116d503 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:49.905481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d40d6-07ad-43b2-b93a-96a81116d503 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:49.914954 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1407/2827] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:49.977106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312aa3-65fe-433c-a6f1-1d6b4e34e366 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:50.039477 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312aa3-65fe-433c-a6f1-1d6b4e34e366 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:50.044706 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:50.045572 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:50.045857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:50.046134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] PATH_INFO: `/v3/users/a04856742a0b4c179d37aa69ddf45c55` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:50.047376 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a04856742a0b4c179d37aa69ddf45c55)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:50.049295 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:50.071308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:50.291750 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a04856742a0b4c179d37aa69ddf45c55'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:50.292408 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a04856742a0b4c179d37aa69ddf45c55'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:50.293608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a04856742a0b4c179d37aa69ddf45c55'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:50.300556 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a04856742a0b4c179d37aa69ddf45c55'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:50.312622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a04856742a0b4c179d37aa69ddf45c55'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:50.407018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1421/2828] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:49 2022] DELETE /identity/v3/users/a04856742a0b4c179d37aa69ddf45c55 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:50.414858 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd36db9-44a0-4a41-b795-975fb01b4d83 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:50.415470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd36db9-44a0-4a41-b795-975fb01b4d83 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:50.415470 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd36db9-44a0-4a41-b795-975fb01b4d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:50.433769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-afd36db9-44a0-4a41-b795-975fb01b4d83 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:50.434976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-afd36db9-44a0-4a41-b795-975fb01b4d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:50.497458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1408/2829] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:50.507200 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:50.508638 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:50.565314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed606dbd-bfad-4977-8478-3ec4bf6b2173 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:50.568842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260de6f0-23a2-4647-9c56-aa8f39e15891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:50.569504 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de6f0-23a2-4647-9c56-aa8f39e15891 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:50.569786 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de6f0-23a2-4647-9c56-aa8f39e15891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:50.570058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de6f0-23a2-4647-9c56-aa8f39e15891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:50.579693 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260de6f0-23a2-4647-9c56-aa8f39e15891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:50.581139 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260de6f0-23a2-4647-9c56-aa8f39e15891 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:50.590901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1422/2830] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:51.042151 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd36db9-44a0-4a41-b795-975fb01b4d83 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:51.043634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-afd36db9-44a0-4a41-b795-975fb01b4d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:51.059213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd36db9-44a0-4a41-b795-975fb01b4d83 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:51.064189 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:51.065167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:51.065519 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:51.065864 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] PATH_INFO: `/v3/projects/da7fcd45e9224a1688fe535254887bdc` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:51.067138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7fcd45e9224a1688fe535254887bdc)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:51.069409 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:51.213809 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:51.222181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7fcd45e9224a1688fe535254887bdc', 'request_id': 'req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.227745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7fcd45e9224a1688fe535254887bdc', 'request_id': 'req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.311123 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7fcd45e9224a1688fe535254887bdc', 'request_id': 'req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.324731 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] The token cache is being invalidate because project da7fcd45e9224a1688fe535254887bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:51.325635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.326782 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.331678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1409/2831] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:51 2022] DELETE /identity/v3/projects/da7fcd45e9224a1688fe535254887bdc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:51.341910 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260de6f0-23a2-4647-9c56-aa8f39e15891 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:51.343800 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260de6f0-23a2-4647-9c56-aa8f39e15891 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:51.411893 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:51.412986 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:51.413377 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:51.413571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] PATH_INFO: `/v3/users/01e2840516964ddd9fa9594c82d76bf6` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:51.414733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e2840516964ddd9fa9594c82d76bf6)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:51.416690 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:51.630352 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:51.635714 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e2840516964ddd9fa9594c82d76bf6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.636457 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e2840516964ddd9fa9594c82d76bf6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.637066 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e2840516964ddd9fa9594c82d76bf6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.644152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e2840516964ddd9fa9594c82d76bf6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.656464 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-b87a5e7a-fb44-408f-be81-220d88fa5dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e2840516964ddd9fa9594c82d76bf6'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:51.737637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1423/2832] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:51 2022] DELETE /identity/v3/users/01e2840516964ddd9fa9594c82d76bf6 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:52.241129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:52.242685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:52.256246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc850a9-b8b6-4b45-8ea8-b0cc57708489 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:52.320596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:52.321717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:52.321992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:52.322265 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] PATH_INFO: `/v3/projects/9409b73f16404eab9dae8cfdef042fd6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:52.323501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=9409b73f16404eab9dae8cfdef042fd6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:52.325882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:52.468386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9409b73f16404eab9dae8cfdef042fd6', 'request_id': 'req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:52.473762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9409b73f16404eab9dae8cfdef042fd6', 'request_id': 'req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:52.549224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9409b73f16404eab9dae8cfdef042fd6', 'request_id': 'req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:52.562074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] The token cache is being invalidate because project 9409b73f16404eab9dae8cfdef042fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:52.562368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:52.563009 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-417bb4c1-a360-4f2d-9d6b-a7957cbe6eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:52.564867 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1410/2833] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:52 2022] DELETE /identity/v3/projects/9409b73f16404eab9dae8cfdef042fd6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:52.569486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37680-9222-4dd4-8f2c-9603d3561fc5 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:52.569967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37680-9222-4dd4-8f2c-9603d3561fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:52.570184 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37680-9222-4dd4-8f2c-9603d3561fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:52.592043 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e7c37680-9222-4dd4-8f2c-9603d3561fc5 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:52.593270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e7c37680-9222-4dd4-8f2c-9603d3561fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:52.687388 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1424/2834] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:52.695497 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa059c6-f8d5-4ed6-b3bf-2dad9b5f9a12 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:52.696056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa059c6-f8d5-4ed6-b3bf-2dad9b5f9a12 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:52.696345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa059c6-f8d5-4ed6-b3bf-2dad9b5f9a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:52.728586 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5aa059c6-f8d5-4ed6-b3bf-2dad9b5f9a12 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:52.729963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5aa059c6-f8d5-4ed6-b3bf-2dad9b5f9a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:52.730154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f868ece-9e4a-49c1-bd4b-bb7e97472012 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:52.730470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f868ece-9e4a-49c1-bd4b-bb7e97472012 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:52.730941 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f868ece-9e4a-49c1-bd4b-bb7e97472012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:52.743403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1411/2835] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:52.748609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa059c6-f8d5-4ed6-b3bf-2dad9b5f9a12 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:52.751820 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-4f868ece-9e4a-49c1-bd4b-bb7e97472012 None None] MFA Rules not processed for user `fccd3de89cd94e5fa4c488c16d993241`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:52.753213 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-4f868ece-9e4a-49c1-bd4b-bb7e97472012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:52.762585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa059c6-f8d5-4ed6-b3bf-2dad9b5f9a12 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:52.768703 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:52.769175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:52.769414 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:52.769786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:52.771246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:00:52.771246 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:52.772349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:52.791299 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1412/2836] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 12:00:52.794592 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:52.807819 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a5d0-e3ef-40fb-8b63-58bc2a176b02 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:52.810433 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7e13b-e6bc-4493-afc2-d0e57ab0abff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:52.811129 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7e13b-e6bc-4493-afc2-d0e57ab0abff admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:52.811453 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7e13b-e6bc-4493-afc2-d0e57ab0abff admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:52.811733 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7e13b-e6bc-4493-afc2-d0e57ab0abff admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:52.812449 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7e13b-e6bc-4493-afc2-d0e57ab0abff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:52.814416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7e13b-e6bc-4493-afc2-d0e57ab0abff admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:52.835475 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1425/2837] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:52.912302 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1413/2838] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:52 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:52.916391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f868ece-9e4a-49c1-bd4b-bb7e97472012 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:52.928168 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f868ece-9e4a-49c1-bd4b-bb7e97472012 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:52.988577 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:52.989258 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:52.989543 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:52.989822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:52.990528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:52.992527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.001799 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:53.154148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1426/2839] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:52 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:53.156686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7e13b-e6bc-4493-afc2-d0e57ab0abff admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.158817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.160563 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:53.167596 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7e13b-e6bc-4493-afc2-d0e57ab0abff admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:53.172308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dec1-43b6-48c1-8bfe-82e2aeba523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:53.172996 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dec1-43b6-48c1-8bfe-82e2aeba523f admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.173277 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dec1-43b6-48c1-8bfe-82e2aeba523f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.173593 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9dec1-43b6-48c1-8bfe-82e2aeba523f admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.174321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9dec1-43b6-48c1-8bfe-82e2aeba523f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:53.175934 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9dec1-43b6-48c1-8bfe-82e2aeba523f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.186615 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1414/2840] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:53.190402 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dec1-43b6-48c1-8bfe-82e2aeba523f admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.195308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:53.201572 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9dec1-43b6-48c1-8bfe-82e2aeba523f admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:53.206147 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a4b14-a2f0-46c1-8558-b153c23fc162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:53.206889 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a4b14-a2f0-46c1-8558-b153c23fc162 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.207127 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a4b14-a2f0-46c1-8558-b153c23fc162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.207450 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a4b14-a2f0-46c1-8558-b153c23fc162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.208292 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a4b14-a2f0-46c1-8558-b153c23fc162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:53.209881 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a4b14-a2f0-46c1-8558-b153c23fc162 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.223071 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd4c13-0d76-48b9-b575-401c22380ab3 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:53.223233 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1415/2841] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 12:00:53.224170 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f3be-ce06-4209-96c0-e732a00dd72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:53.225346 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a4b14-a2f0-46c1-8558-b153c23fc162 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.225450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f3be-ce06-4209-96c0-e732a00dd72b service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.225450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f3be-ce06-4209-96c0-e732a00dd72b service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.225450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f3be-ce06-4209-96c0-e732a00dd72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.227203 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2323f3be-ce06-4209-96c0-e732a00dd72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:53.234074 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a4b14-a2f0-46c1-8558-b153c23fc162 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:53.236999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323f3be-ce06-4209-96c0-e732a00dd72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:53.238416 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323f3be-ce06-4209-96c0-e732a00dd72b service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.238992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e88a-90b9-45fa-a418-d4819231364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:53.239913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e88a-90b9-45fa-a418-d4819231364a admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.240204 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e88a-90b9-45fa-a418-d4819231364a admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.240487 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e88a-90b9-45fa-a418-d4819231364a admin admin] PATH_INFO: `/v3/projects/f1517b6c6094479898491a9efee0172b/users/2871ece547b04c2f9c8c89b6d278831d/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.246909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1427/2842] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:53.256419 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6e88a-90b9-45fa-a418-d4819231364a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1517b6c6094479898491a9efee0172b, user_id=2871ece547b04c2f9c8c89b6d278831d, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:53.259225 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6e88a-90b9-45fa-a418-d4819231364a admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.315903 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1416/2843] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:53 2022] PUT /identity/v3/projects/f1517b6c6094479898491a9efee0172b/users/2871ece547b04c2f9c8c89b6d278831d/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:53.328010 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28ddbc-506e-4c09-88ca-734cdba4b80c None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.328333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28ddbc-506e-4c09-88ca-734cdba4b80c None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.328616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28ddbc-506e-4c09-88ca-734cdba4b80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.354949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-cb28ddbc-506e-4c09-88ca-734cdba4b80c None None] MFA Rules not processed for user `2871ece547b04c2f9c8c89b6d278831d`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:53.356228 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-cb28ddbc-506e-4c09-88ca-734cdba4b80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:53.423618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1428/2844] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:53.428295 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e88a-90b9-45fa-a418-d4819231364a admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.494669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e88a-90b9-45fa-a418-d4819231364a admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:53.500585 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355de583-2146-4de2-bdda-07f23e7da84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:53.501240 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355de583-2146-4de2-bdda-07f23e7da84b admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.501556 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355de583-2146-4de2-bdda-07f23e7da84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.501823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355de583-2146-4de2-bdda-07f23e7da84b admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.502510 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355de583-2146-4de2-bdda-07f23e7da84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:53.504427 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355de583-2146-4de2-bdda-07f23e7da84b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.582040 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1417/2845] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:53 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:53.586333 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28ddbc-506e-4c09-88ca-734cdba4b80c None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.597063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28ddbc-506e-4c09-88ca-734cdba4b80c None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:53.654329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478691-fb2c-40bb-8593-bd5c9b5a4e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:53.654991 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478691-fb2c-40bb-8593-bd5c9b5a4e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.655296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478691-fb2c-40bb-8593-bd5c9b5a4e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.655574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42478691-fb2c-40bb-8593-bd5c9b5a4e00 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.656616 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42478691-fb2c-40bb-8593-bd5c9b5a4e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:53.658522 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42478691-fb2c-40bb-8593-bd5c9b5a4e00 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.750104 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1429/2846] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:53 2022] POST /identity/v3/users => generated 296 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:53.754192 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355de583-2146-4de2-bdda-07f23e7da84b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.764554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355de583-2146-4de2-bdda-07f23e7da84b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:53.769229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef33ba4-0a5f-4ec7-bf7d-9bbf62a0233b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:53.769891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef33ba4-0a5f-4ec7-bf7d-9bbf62a0233b admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.770167 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef33ba4-0a5f-4ec7-bf7d-9bbf62a0233b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.770469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef33ba4-0a5f-4ec7-bf7d-9bbf62a0233b admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.771220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef33ba4-0a5f-4ec7-bf7d-9bbf62a0233b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:53.772811 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef33ba4-0a5f-4ec7-bf7d-9bbf62a0233b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.783539 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1418/2847] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:53.787709 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478691-fb2c-40bb-8593-bd5c9b5a4e00 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.798981 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42478691-fb2c-40bb-8593-bd5c9b5a4e00 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:53.803842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2013b-01b0-47fe-b5ec-726d86530625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:53.804746 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2013b-01b0-47fe-b5ec-726d86530625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:53.805033 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2013b-01b0-47fe-b5ec-726d86530625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:53.805312 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2013b-01b0-47fe-b5ec-726d86530625 admin admin] PATH_INFO: `/v3/projects/37bbdc3e1598424fb6bdc4923b6baf6a/users/a3f0ef79fe1148608bd614ec63a06d64/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:53.818231 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:53.822047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2013b-01b0-47fe-b5ec-726d86530625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bbdc3e1598424fb6bdc4923b6baf6a, user_id=a3f0ef79fe1148608bd614ec63a06d64, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:53.824760 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2013b-01b0-47fe-b5ec-726d86530625 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:53.911495 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1430/2848] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:53 2022] PUT /identity/v3/projects/37bbdc3e1598424fb6bdc4923b6baf6a/users/a3f0ef79fe1148608bd614ec63a06d64/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:53.918103 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef33ba4-0a5f-4ec7-bf7d-9bbf62a0233b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:53.928584 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef33ba4-0a5f-4ec7-bf7d-9bbf62a0233b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:54.012321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e0c34-5960-41e9-b7e0-3dfeb3941154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:54.013179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e0c34-5960-41e9-b7e0-3dfeb3941154 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:54.013564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e0c34-5960-41e9-b7e0-3dfeb3941154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:54.013940 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e0c34-5960-41e9-b7e0-3dfeb3941154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:54.015635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e0c34-5960-41e9-b7e0-3dfeb3941154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:54.017424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e0c34-5960-41e9-b7e0-3dfeb3941154 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:54.032089 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1419/2849] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:54.036497 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2013b-01b0-47fe-b5ec-726d86530625 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:54.048785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2013b-01b0-47fe-b5ec-726d86530625 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:54.053618 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1053e-ef3a-457b-9cc2-f346fbe83ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:54.054553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1053e-ef3a-457b-9cc2-f346fbe83ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:54.055006 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1053e-ef3a-457b-9cc2-f346fbe83ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:54.055445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1053e-ef3a-457b-9cc2-f346fbe83ea7 admin admin] PATH_INFO: `/v3/projects/37bbdc3e1598424fb6bdc4923b6baf6a/users/a3f0ef79fe1148608bd614ec63a06d64/roles/007af6bd1b734517b934b4cac45faf2c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:54.059136 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1053e-ef3a-457b-9cc2-f346fbe83ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bbdc3e1598424fb6bdc4923b6baf6a, user_id=a3f0ef79fe1148608bd614ec63a06d64, role_id=007af6bd1b734517b934b4cac45faf2c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:54.063725 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1053e-ef3a-457b-9cc2-f346fbe83ea7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:54.075199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1431/2850] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:54 2022] PUT /identity/v3/projects/37bbdc3e1598424fb6bdc4923b6baf6a/users/a3f0ef79fe1148608bd614ec63a06d64/roles/007af6bd1b734517b934b4cac45faf2c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:54.085170 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe02736-d09b-492c-9095-d809edf85ccb None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:54.085463 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe02736-d09b-492c-9095-d809edf85ccb None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:54.085741 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe02736-d09b-492c-9095-d809edf85ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:54.116564 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cbe02736-d09b-492c-9095-d809edf85ccb None None] MFA Rules not processed for user `a3f0ef79fe1148608bd614ec63a06d64`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:54.117964 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cbe02736-d09b-492c-9095-d809edf85ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:54.143774 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1053e-ef3a-457b-9cc2-f346fbe83ea7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:54.150138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:54.207717 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1420/2851] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:54.213108 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1053e-ef3a-457b-9cc2-f346fbe83ea7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:54.217809 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:54.218593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:54.218868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:54.219148 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:54.223474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe02736-d09b-492c-9095-d809edf85ccb None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:54.223588 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:54.223678 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cbe02736-d09b-492c-9095-d809edf85ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:54.283548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe02736-d09b-492c-9095-d809edf85ccb None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:54.287317 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947efa16-8a44-40c6-840e-3e6ec287c20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:54.287963 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947efa16-8a44-40c6-840e-3e6ec287c20f service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:54.288243 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947efa16-8a44-40c6-840e-3e6ec287c20f service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:54.288515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947efa16-8a44-40c6-840e-3e6ec287c20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:54.293756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:54.295215 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:54.299064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947efa16-8a44-40c6-840e-3e6ec287c20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:54.300501 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947efa16-8a44-40c6-840e-3e6ec287c20f service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:54.305400 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1432/2852] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:54.310230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1421/2853] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 12:00:54.944289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:55.011013 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdebbc-d691-46c6-b411-e53e02a058aa service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:55.016301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:55.017182 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:55.017465 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:55.017785 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] PATH_INFO: `/v3/users/2871ece547b04c2f9c8c89b6d278831d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:55.019174 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2871ece547b04c2f9c8c89b6d278831d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:55.021140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:55.253926 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:55.268068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2871ece547b04c2f9c8c89b6d278831d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:55.268730 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2871ece547b04c2f9c8c89b6d278831d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:55.269335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2871ece547b04c2f9c8c89b6d278831d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:55.276328 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2871ece547b04c2f9c8c89b6d278831d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:55.288225 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2871ece547b04c2f9c8c89b6d278831d'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:55.385372 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1433/2854] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:54 2022] DELETE /identity/v3/users/2871ece547b04c2f9c8c89b6d278831d => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:55.390015 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457330b-5884-41b2-a228-d8d7fd144fd6 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:55.390476 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457330b-5884-41b2-a228-d8d7fd144fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:55.390625 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457330b-5884-41b2-a228-d8d7fd144fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:55.407728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c457330b-5884-41b2-a228-d8d7fd144fd6 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:55.408939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c457330b-5884-41b2-a228-d8d7fd144fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:55.470791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1422/2855] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 12:00:55.481044 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:55.482694 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:55.543494 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5926e2-b38a-4d26-8d3f-6a1eeff09fd5 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:55.547172 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e27670-33a1-478e-807e-0abbed42ea08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:55.547891 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e27670-33a1-478e-807e-0abbed42ea08 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:55.548181 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e27670-33a1-478e-807e-0abbed42ea08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:55.548461 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e27670-33a1-478e-807e-0abbed42ea08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:55.558232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e27670-33a1-478e-807e-0abbed42ea08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:55.559646 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e27670-33a1-478e-807e-0abbed42ea08 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:55.569665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1434/2856] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:00:56.263143 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457330b-5884-41b2-a228-d8d7fd144fd6 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:56.264692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c457330b-5884-41b2-a228-d8d7fd144fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:56.284852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457330b-5884-41b2-a228-d8d7fd144fd6 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:56.292203 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:56.294791 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:56.295330 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:56.295842 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] PATH_INFO: `/v3/projects/f1517b6c6094479898491a9efee0172b` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:56.305071 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1517b6c6094479898491a9efee0172b)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:56.307033 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:56.431590 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1517b6c6094479898491a9efee0172b', 'request_id': 'req-31447ced-995b-4195-a178-94bd39043baa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.441353 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1517b6c6094479898491a9efee0172b', 'request_id': 'req-31447ced-995b-4195-a178-94bd39043baa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.444368 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:56.522542 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1517b6c6094479898491a9efee0172b', 'request_id': 'req-31447ced-995b-4195-a178-94bd39043baa'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.535902 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] The token cache is being invalidate because project f1517b6c6094479898491a9efee0172b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:56.536197 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.536913 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.538577 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1423/2857] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:56 2022] DELETE /identity/v3/projects/f1517b6c6094479898491a9efee0172b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:56.542665 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e27670-33a1-478e-807e-0abbed42ea08 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:56.555446 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e27670-33a1-478e-807e-0abbed42ea08 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:56.629339 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:56.631275 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:56.631747 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:56.632264 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] PATH_INFO: `/v3/users/a3f0ef79fe1148608bd614ec63a06d64` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:56.634067 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f0ef79fe1148608bd614ec63a06d64)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:56.636911 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:56.842364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f0ef79fe1148608bd614ec63a06d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.843061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f0ef79fe1148608bd614ec63a06d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.843895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f0ef79fe1148608bd614ec63a06d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.851004 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:56.856124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f0ef79fe1148608bd614ec63a06d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.872134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-145b9385-911f-4349-a2e0-e206da863356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f0ef79fe1148608bd614ec63a06d64'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:56.944492 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1435/2858] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:00:56 2022] DELETE /identity/v3/users/a3f0ef79fe1148608bd614ec63a06d64 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:00:57.556668 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:57.558195 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:57.594311 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31447ced-995b-4195-a178-94bd39043baa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:57.676471 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:57.677458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:57.677786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:57.678226 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] PATH_INFO: `/v3/projects/37bbdc3e1598424fb6bdc4923b6baf6a` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:57.679423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37bbdc3e1598424fb6bdc4923b6baf6a)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:57.681617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:57.797142 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37bbdc3e1598424fb6bdc4923b6baf6a', 'request_id': 'req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:57.806136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37bbdc3e1598424fb6bdc4923b6baf6a', 'request_id': 'req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:57.899831 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37bbdc3e1598424fb6bdc4923b6baf6a', 'request_id': 'req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:57.911715 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] The token cache is being invalidate because project 37bbdc3e1598424fb6bdc4923b6baf6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:00:57.912001 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:57.912617 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-99c782bd-615f-4dd2-97fa-5b03f2bff6e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:00:57.914187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1424/2859] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:00:57 2022] DELETE /identity/v3/projects/37bbdc3e1598424fb6bdc4923b6baf6a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:57.919056 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214c427-c7a2-4354-a47d-83b70d8e4212 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:57.919582 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214c427-c7a2-4354-a47d-83b70d8e4212 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:57.919672 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214c427-c7a2-4354-a47d-83b70d8e4212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:57.940192 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-6214c427-c7a2-4354-a47d-83b70d8e4212 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:57.941441 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-6214c427-c7a2-4354-a47d-83b70d8e4212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:58.014362 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1436/2860] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:58.023593 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa749-2843-4c43-9ebb-3596c4a758b8 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.024137 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa749-2843-4c43-9ebb-3596c4a758b8 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.024794 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fa749-2843-4c43-9ebb-3596c4a758b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.047774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-8d8fa749-2843-4c43-9ebb-3596c4a758b8 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:58.049000 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8d8fa749-2843-4c43-9ebb-3596c4a758b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:58.066653 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1425/2861] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:58.067024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214c427-c7a2-4354-a47d-83b70d8e4212 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:58.074354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214c427-c7a2-4354-a47d-83b70d8e4212 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:58.080865 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:58.081559 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.081907 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.082211 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.083154 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:00:58.083528 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:58.085238 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:58.100654 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1437/2862] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 12:00:58.105840 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fa749-2843-4c43-9ebb-3596c4a758b8 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:58.116380 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fa749-2843-4c43-9ebb-3596c4a758b8 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:58.122053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4faa3d-a4da-4401-a395-67bba03c89b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:58.122663 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4faa3d-a4da-4401-a395-67bba03c89b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.122948 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4faa3d-a4da-4401-a395-67bba03c89b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.123217 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4faa3d-a4da-4401-a395-67bba03c89b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.123957 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4faa3d-a4da-4401-a395-67bba03c89b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:58.125937 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4faa3d-a4da-4401-a395-67bba03c89b6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:58.313743 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1426/2863] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:00:58 2022] POST /identity/v3/projects => generated 367 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 12:00:58.318268 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:58.331371 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211e09c-1f2b-4b11-9c7f-355c45866af7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:58.397768 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a9d1-eb58-4faa-944c-7eb55f026854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:58.398450 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a9d1-eb58-4faa-944c-7eb55f026854 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.403999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a9d1-eb58-4faa-944c-7eb55f026854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.403999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14a9d1-eb58-4faa-944c-7eb55f026854 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.403999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14a9d1-eb58-4faa-944c-7eb55f026854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:58.403999 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14a9d1-eb58-4faa-944c-7eb55f026854 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:58.487406 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1438/2864] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:00:58 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:00:58.491179 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4faa3d-a4da-4401-a395-67bba03c89b6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:58.502139 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4faa3d-a4da-4401-a395-67bba03c89b6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:58.506828 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385d4f6-1870-43fc-a290-dc37757d0afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:58.507531 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385d4f6-1870-43fc-a290-dc37757d0afa admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.507815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385d4f6-1870-43fc-a290-dc37757d0afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.508091 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385d4f6-1870-43fc-a290-dc37757d0afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.508820 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385d4f6-1870-43fc-a290-dc37757d0afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:58.510412 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385d4f6-1870-43fc-a290-dc37757d0afa admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:58.522147 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1427/2865] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:58.526046 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a9d1-eb58-4faa-944c-7eb55f026854 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:58.537103 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14a9d1-eb58-4faa-944c-7eb55f026854 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:58.541949 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a5fe5-efbe-488b-b528-9717c03c1ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:58.543199 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a5fe5-efbe-488b-b528-9717c03c1ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.543520 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a5fe5-efbe-488b-b528-9717c03c1ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.543802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a5fe5-efbe-488b-b528-9717c03c1ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.544575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a5fe5-efbe-488b-b528-9717c03c1ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:58.546140 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a5fe5-efbe-488b-b528-9717c03c1ff7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:58.556878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1439/2866] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:00:58.560949 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385d4f6-1870-43fc-a290-dc37757d0afa admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:58.571692 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385d4f6-1870-43fc-a290-dc37757d0afa admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:58.577181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:58.578469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.578469 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.581216 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] PATH_INFO: `/v3/projects/ce9abc01568f41e196913da77e753a7f/users/b8d8cd28030c43ec8c42f06809a28f5c/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.597984 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9abc01568f41e196913da77e753a7f, user_id=b8d8cd28030c43ec8c42f06809a28f5c, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:58.601062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:58.705815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1428/2867] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:00:58 2022] PUT /identity/v3/projects/ce9abc01568f41e196913da77e753a7f/users/b8d8cd28030c43ec8c42f06809a28f5c/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:00:58.716421 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c411a-a0a6-4833-8d45-a4429a986e25 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.716718 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c411a-a0a6-4833-8d45-a4429a986e25 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.717002 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c411a-a0a6-4833-8d45-a4429a986e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.746573 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-2e7c411a-a0a6-4833-8d45-a4429a986e25 None None] MFA Rules not processed for user `b8d8cd28030c43ec8c42f06809a28f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:00:58.747705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2e7c411a-a0a6-4833-8d45-a4429a986e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:58.790060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:58.834244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1440/2868] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:00:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:00:58.848112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:00:58.850036 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:00:58.872252 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:00:58.941616 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac28b50-3476-45bb-b3bd-7c2c240ccbef admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:00:58.945566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e939bc73c40140cb89517e918722d154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:00:58.946481 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:00:58.946763 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:00:58.947100 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:00:58.956354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:00:58.957755 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:00:58.967045 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1429/2869] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:00:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:05.176233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c411a-a0a6-4833-8d45-a4429a986e25 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:05.178872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-2e7c411a-a0a6-4833-8d45-a4429a986e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:05.257642 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c411a-a0a6-4833-8d45-a4429a986e25 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:05.262569 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:05.263224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:05.263538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:05.263819 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:05.265575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:05.276878 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:05.278517 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:05.289905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1441/2870] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:06.043177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:06.044686 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:06.120670 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940142f4-2ad2-4ad7-8aa4-1c583eb7fde2 service glance] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:06.126334 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:06.127536 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:06.127710 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:06.127991 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] PATH_INFO: `/v3/users/06324752e1c043c2a4f34d3751efbcfb` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:06.129230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=06324752e1c043c2a4f34d3751efbcfb)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:06.131589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:06.303019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06324752e1c043c2a4f34d3751efbcfb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.303789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06324752e1c043c2a4f34d3751efbcfb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.304396 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06324752e1c043c2a4f34d3751efbcfb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.313778 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06324752e1c043c2a4f34d3751efbcfb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.328051 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06324752e1c043c2a4f34d3751efbcfb'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.437789 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1430/2871] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:01:06 2022] DELETE /identity/v3/users/06324752e1c043c2a4f34d3751efbcfb => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:06.701209 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:06.711681 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690acff-4754-466e-bf29-2bf9f35ab735 service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:06.778232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:06.779036 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:06.779882 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:06.780179 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] PATH_INFO: `/v3/projects/e2d0c9baa584498494257302e9d5d98d` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:06.781782 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d0c9baa584498494257302e9d5d98d)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:06.783969 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:06.893781 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d0c9baa584498494257302e9d5d98d', 'request_id': 'req-06f0590d-f170-4050-a2c6-c35271245817'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.898872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:06.900253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d0c9baa584498494257302e9d5d98d', 'request_id': 'req-06f0590d-f170-4050-a2c6-c35271245817'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.971348 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d0c9baa584498494257302e9d5d98d', 'request_id': 'req-06f0590d-f170-4050-a2c6-c35271245817'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.985261 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] The token cache is being invalidate because project e2d0c9baa584498494257302e9d5d98d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:01:06.985552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.986224 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:06.987805 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1442/2872] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:01:06 2022] DELETE /identity/v3/projects/e2d0c9baa584498494257302e9d5d98d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:06.991774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:06.993177 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:07.006656 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74036f20-3d14-43d2-b284-5b6a88688bff admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:07.020966 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:07.066882 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:07.073394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:07.073394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:07.073394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] PATH_INFO: `/v3/users/189c710a05d34f2e8e87069c7377e6cf` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:07.073394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=189c710a05d34f2e8e87069c7377e6cf)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:07.073394 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:07.287308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '189c710a05d34f2e8e87069c7377e6cf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:07.288090 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '189c710a05d34f2e8e87069c7377e6cf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:07.288823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '189c710a05d34f2e8e87069c7377e6cf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:07.296918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '189c710a05d34f2e8e87069c7377e6cf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:07.309764 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-305ae737-a942-48a7-b641-84ec0c4f7ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '189c710a05d34f2e8e87069c7377e6cf'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:07.370284 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1431/2873] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:01:06 2022] DELETE /identity/v3/users/189c710a05d34f2e8e87069c7377e6cf => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:07.695370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:07.697097 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:07.711207 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0590d-f170-4050-a2c6-c35271245817 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:07.772957 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:07.773828 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:07.774111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:07.774386 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] PATH_INFO: `/v3/projects/d5983a7bc2fe4427b7bc24ea41c65eda` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:07.775701 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5983a7bc2fe4427b7bc24ea41c65eda)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:07.777983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:08.041734 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5983a7bc2fe4427b7bc24ea41c65eda', 'request_id': 'req-36738f14-5634-4b9b-ad2c-f38726178315'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:08.055278 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5983a7bc2fe4427b7bc24ea41c65eda', 'request_id': 'req-36738f14-5634-4b9b-ad2c-f38726178315'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:08.127603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5983a7bc2fe4427b7bc24ea41c65eda', 'request_id': 'req-36738f14-5634-4b9b-ad2c-f38726178315'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:08.139442 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] The token cache is being invalidate because project d5983a7bc2fe4427b7bc24ea41c65eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:01:08.139731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:08.140391 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-36738f14-5634-4b9b-ad2c-f38726178315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:08.141987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1443/2874] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:01:07 2022] DELETE /identity/v3/projects/d5983a7bc2fe4427b7bc24ea41c65eda => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:08.150883 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a2da2-5b55-46ae-90ab-07f49f3b8b7b None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.151438 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a2da2-5b55-46ae-90ab-07f49f3b8b7b None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.151535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a2da2-5b55-46ae-90ab-07f49f3b8b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.169164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-fb0a2da2-5b55-46ae-90ab-07f49f3b8b7b None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:08.170352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-fb0a2da2-5b55-46ae-90ab-07f49f3b8b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:08.231475 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1432/2875] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:08.239571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3305cf4-0bbe-40cb-8218-d32477b1e1bb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.239985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3305cf4-0bbe-40cb-8218-d32477b1e1bb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.240373 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3305cf4-0bbe-40cb-8218-d32477b1e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.260980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-e3305cf4-0bbe-40cb-8218-d32477b1e1bb None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:08.262183 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-e3305cf4-0bbe-40cb-8218-d32477b1e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:08.269866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1444/2876] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:08.275065 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a2da2-5b55-46ae-90ab-07f49f3b8b7b None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:08.285849 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a2da2-5b55-46ae-90ab-07f49f3b8b7b None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:08.290473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:08.291136 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.291466 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.291749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.292535 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:01:08.292848 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:08.294458 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:08.306666 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1433/2877] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:01:08.311552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3305cf4-0bbe-40cb-8218-d32477b1e1bb None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:08.321335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3305cf4-0bbe-40cb-8218-d32477b1e1bb None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:08.325861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80849605-14ca-4acb-8149-4bb55b23dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:08.326508 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80849605-14ca-4acb-8149-4bb55b23dd0b admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.326789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80849605-14ca-4acb-8149-4bb55b23dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.327063 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80849605-14ca-4acb-8149-4bb55b23dd0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.327790 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80849605-14ca-4acb-8149-4bb55b23dd0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:08.329661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80849605-14ca-4acb-8149-4bb55b23dd0b admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:08.415219 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1445/2878] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:01:08 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:01:08.419473 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:08.430549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ab984-b0df-4d89-bbf7-5b61b0ed83c0 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:08.451354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:08.487911 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75480a46-a30f-4d7a-8407-e36fd38d1afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:08.488500 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75480a46-a30f-4d7a-8407-e36fd38d1afe admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.488784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75480a46-a30f-4d7a-8407-e36fd38d1afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.489064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75480a46-a30f-4d7a-8407-e36fd38d1afe admin admin] PATH_INFO: `/v3/users` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.489754 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75480a46-a30f-4d7a-8407-e36fd38d1afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:08.491749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75480a46-a30f-4d7a-8407-e36fd38d1afe admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:08.619868 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1434/2879] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:01:08 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:01:08.624024 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80849605-14ca-4acb-8149-4bb55b23dd0b admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:08.634053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80849605-14ca-4acb-8149-4bb55b23dd0b admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:08.638590 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46cc5a-98f8-4dd0-a321-5952f05a4105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:08.639240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46cc5a-98f8-4dd0-a321-5952f05a4105 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.640107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46cc5a-98f8-4dd0-a321-5952f05a4105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.640334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46cc5a-98f8-4dd0-a321-5952f05a4105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.641051 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46cc5a-98f8-4dd0-a321-5952f05a4105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:08.642619 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46cc5a-98f8-4dd0-a321-5952f05a4105 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:08.653217 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1446/2880] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:01:08.657936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75480a46-a30f-4d7a-8407-e36fd38d1afe admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:08.668321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75480a46-a30f-4d7a-8407-e36fd38d1afe admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:08.673026 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4d098-c169-4463-ab29-683e30ddfb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:08.673680 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4d098-c169-4463-ab29-683e30ddfb6d admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.673967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4d098-c169-4463-ab29-683e30ddfb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.674248 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4d098-c169-4463-ab29-683e30ddfb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.674976 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4d098-c169-4463-ab29-683e30ddfb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:08.676613 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4d098-c169-4463-ab29-683e30ddfb6d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:08.687303 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1435/2881] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:01:08.691107 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46cc5a-98f8-4dd0-a321-5952f05a4105 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:08.702462 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46cc5a-98f8-4dd0-a321-5952f05a4105 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:08.706920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:08.707868 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.708155 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.708431 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] PATH_INFO: `/v3/projects/be67580a1ce1429ab40e662e244567d6/users/c11b94185b4d4e2a8ce6f9a27fdf39e3/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.722847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be67580a1ce1429ab40e662e244567d6, user_id=c11b94185b4d4e2a8ce6f9a27fdf39e3, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:08.726498 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:08.819531 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1447/2882] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:01:08 2022] PUT /identity/v3/projects/be67580a1ce1429ab40e662e244567d6/users/c11b94185b4d4e2a8ce6f9a27fdf39e3/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:08.830566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1011a-7e41-4c55-aa9d-2d3af88ba13d None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:08.830866 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1011a-7e41-4c55-aa9d-2d3af88ba13d None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:08.831152 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1011a-7e41-4c55-aa9d-2d3af88ba13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:08.861762 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-bbb1011a-7e41-4c55-aa9d-2d3af88ba13d None None] MFA Rules not processed for user `c11b94185b4d4e2a8ce6f9a27fdf39e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:08.862982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bbb1011a-7e41-4c55-aa9d-2d3af88ba13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:08.939324 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1436/2883] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:01:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:08.946920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:08.948637 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:08.964096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:09.009761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3b3e3-be2f-4002-8a82-c9ae4ed42ca0 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:09.016086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e27b13-66c6-4002-81f5-0d6576aba719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:09.016086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e27b13-66c6-4002-81f5-0d6576aba719 service cinder] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:09.016086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e27b13-66c6-4002-81f5-0d6576aba719 service cinder] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:09.016086 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e27b13-66c6-4002-81f5-0d6576aba719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:09.023909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e27b13-66c6-4002-81f5-0d6576aba719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:09.025269 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e27b13-66c6-4002-81f5-0d6576aba719 service cinder] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:09.034484 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1448/2884] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:11.834640 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1011a-7e41-4c55-aa9d-2d3af88ba13d None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:11.836159 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-bbb1011a-7e41-4c55-aa9d-2d3af88ba13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:11.891372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1011a-7e41-4c55-aa9d-2d3af88ba13d None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:11.896873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84419b6e-55d7-4a4d-b8ae-3c1ff43d4d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:11.897482 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84419b6e-55d7-4a4d-b8ae-3c1ff43d4d6d service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:11.897769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84419b6e-55d7-4a4d-b8ae-3c1ff43d4d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:11.898049 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84419b6e-55d7-4a4d-b8ae-3c1ff43d4d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:11.899925 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-84419b6e-55d7-4a4d-b8ae-3c1ff43d4d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:11.946649 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:11.963230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84419b6e-55d7-4a4d-b8ae-3c1ff43d4d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:11.964701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84419b6e-55d7-4a4d-b8ae-3c1ff43d4d6d service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:11.974566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1437/2885] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:19.810553 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=21 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:22.050821 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=8 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:31.870658 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e27b13-66c6-4002-81f5-0d6576aba719 service cinder] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:31.932521 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e27b13-66c6-4002-81f5-0d6576aba719 service cinder] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:31.937018 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:31.937887 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:31.938176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:31.938471 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] PATH_INFO: `/v3/users/c11b94185b4d4e2a8ce6f9a27fdf39e3` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:31.939671 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11b94185b4d4e2a8ce6f9a27fdf39e3)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:31.941578 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:32.159060 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11b94185b4d4e2a8ce6f9a27fdf39e3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:32.159895 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11b94185b4d4e2a8ce6f9a27fdf39e3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:32.160780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11b94185b4d4e2a8ce6f9a27fdf39e3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:32.168169 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11b94185b4d4e2a8ce6f9a27fdf39e3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:32.180802 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11b94185b4d4e2a8ce6f9a27fdf39e3'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:32.262423 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1449/2886] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:01:31 2022] DELETE /identity/v3/users/c11b94185b4d4e2a8ce6f9a27fdf39e3 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:32.266887 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25402939-27a2-4b8d-995c-7e62f17ad8fe None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:32.267411 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25402939-27a2-4b8d-995c-7e62f17ad8fe None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:32.267509 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25402939-27a2-4b8d-995c-7e62f17ad8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:32.285367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-25402939-27a2-4b8d-995c-7e62f17ad8fe None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:32.287337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-25402939-27a2-4b8d-995c-7e62f17ad8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:32.354818 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1438/2887] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:32.364935 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:32.366327 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:32.422617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c982a-a39f-4fe7-9691-e44aacd37553 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:32.426677 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:32.427458 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:32.427753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:32.428034 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:32.437068 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:32.438486 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:32.448552 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1450/2888] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:32.860737 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25402939-27a2-4b8d-995c-7e62f17ad8fe None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:32.862214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-25402939-27a2-4b8d-995c-7e62f17ad8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:32.875550 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25402939-27a2-4b8d-995c-7e62f17ad8fe None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:32.880480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:32.881543 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:32.881892 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:32.882268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] PATH_INFO: `/v3/projects/be67580a1ce1429ab40e662e244567d6` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:32.883711 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be67580a1ce1429ab40e662e244567d6)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:32.885771 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:33.100772 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be67580a1ce1429ab40e662e244567d6', 'request_id': 'req-bfed5961-2392-4526-9a1e-56f69e002358'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:33.106224 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be67580a1ce1429ab40e662e244567d6', 'request_id': 'req-bfed5961-2392-4526-9a1e-56f69e002358'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:33.185496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be67580a1ce1429ab40e662e244567d6', 'request_id': 'req-bfed5961-2392-4526-9a1e-56f69e002358'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:33.197651 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] The token cache is being invalidate because project be67580a1ce1429ab40e662e244567d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:01:33.197942 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:33.198562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-bfed5961-2392-4526-9a1e-56f69e002358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:33.200280 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1439/2889] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:01:32 2022] DELETE /identity/v3/projects/be67580a1ce1429ab40e662e244567d6 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:01:42.444208 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:42.519470 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:42.520911 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:42.533867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c1613-7168-46da-ae7f-0069632721b3 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:42.592780 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:42.593641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:42.593923 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:42.594201 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] PATH_INFO: `/v3/users/b8d8cd28030c43ec8c42f06809a28f5c` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:42.595437 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d8cd28030c43ec8c42f06809a28f5c)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:42.597341 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:42.781687 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d8cd28030c43ec8c42f06809a28f5c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:42.782270 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d8cd28030c43ec8c42f06809a28f5c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:42.782875 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d8cd28030c43ec8c42f06809a28f5c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:42.789867 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d8cd28030c43ec8c42f06809a28f5c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:42.806967 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d8cd28030c43ec8c42f06809a28f5c'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:42.873985 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1451/2890] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:01:42 2022] DELETE /identity/v3/users/b8d8cd28030c43ec8c42f06809a28f5c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:42.878352 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f82ccd-834d-4428-b160-e00efc9c6627 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:42.878785 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f82ccd-834d-4428-b160-e00efc9c6627 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:42.883825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f82ccd-834d-4428-b160-e00efc9c6627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:42.896423 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-c3f82ccd-834d-4428-b160-e00efc9c6627 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:42.897634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c3f82ccd-834d-4428-b160-e00efc9c6627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:42.953667 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1440/2891] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:42.963053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:42.964600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:43.020630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f925aa0-c7bd-47ac-b063-bed0dcf7b4a4 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:43.024370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf657e3-c2a7-46ee-8ad5-96878c37cd58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:43.025047 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf657e3-c2a7-46ee-8ad5-96878c37cd58 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:43.025330 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf657e3-c2a7-46ee-8ad5-96878c37cd58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:43.025603 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf657e3-c2a7-46ee-8ad5-96878c37cd58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:43.034728 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf657e3-c2a7-46ee-8ad5-96878c37cd58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:43.036212 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf657e3-c2a7-46ee-8ad5-96878c37cd58 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:43.046305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1452/2892] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:43.439634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f82ccd-834d-4428-b160-e00efc9c6627 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:43.442968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-c3f82ccd-834d-4428-b160-e00efc9c6627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:43.456944 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f82ccd-834d-4428-b160-e00efc9c6627 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:43.464951 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:43.465747 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:43.466030 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:43.466375 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] PATH_INFO: `/v3/projects/ce9abc01568f41e196913da77e753a7f` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:43.467521 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9abc01568f41e196913da77e753a7f)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:43.469416 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:43.496301 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:43.589735 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9abc01568f41e196913da77e753a7f', 'request_id': 'req-a561a903-5955-4400-9ea1-9c3f69ca3a31'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:43.595143 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9abc01568f41e196913da77e753a7f', 'request_id': 'req-a561a903-5955-4400-9ea1-9c3f69ca3a31'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:43.645349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9abc01568f41e196913da77e753a7f', 'request_id': 'req-a561a903-5955-4400-9ea1-9c3f69ca3a31'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:43.657056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] The token cache is being invalidate because project ce9abc01568f41e196913da77e753a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:01:43.657348 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:43.657968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-a561a903-5955-4400-9ea1-9c3f69ca3a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:01:43.659552 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1441/2893] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:01:43 2022] DELETE /identity/v3/projects/ce9abc01568f41e196913da77e753a7f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:43.664823 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb320801-730b-4528-9e3f-9f659527a663 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:43.665301 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb320801-730b-4528-9e3f-9f659527a663 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:43.665411 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb320801-730b-4528-9e3f-9f659527a663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:43.682496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-fb320801-730b-4528-9e3f-9f659527a663 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:43.686230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-fb320801-730b-4528-9e3f-9f659527a663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:43.701032 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:43.741983 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1453/2894] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:43.749413 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f963d-251c-4e0d-9519-86a29ae2e793 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:43.749709 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f963d-251c-4e0d-9519-86a29ae2e793 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:43.749989 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f963d-251c-4e0d-9519-86a29ae2e793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:43.768165 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-4e8f963d-251c-4e0d-9519-86a29ae2e793 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:43.769367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-4e8f963d-251c-4e0d-9519-86a29ae2e793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:43.777876 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1442/2895] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:43.781859 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb320801-730b-4528-9e3f-9f659527a663 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:43.791296 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb320801-730b-4528-9e3f-9f659527a663 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:43.798106 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:43.798759 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:43.799049 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:43.799359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:43.800084 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:01:43.800395 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:43.802050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:43.814285 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1454/2896] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:01:43.818645 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f963d-251c-4e0d-9519-86a29ae2e793 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:43.829908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f963d-251c-4e0d-9519-86a29ae2e793 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:43.837454 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df400019-5dd5-4b3f-b364-b9f86afc0bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:43.838106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df400019-5dd5-4b3f-b364-b9f86afc0bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:43.838389 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df400019-5dd5-4b3f-b364-b9f86afc0bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:43.838664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df400019-5dd5-4b3f-b364-b9f86afc0bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:43.839403 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df400019-5dd5-4b3f-b364-b9f86afc0bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:43.841337 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df400019-5dd5-4b3f-b364-b9f86afc0bc6 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:43.941366 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1443/2897] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:01:43 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:01:43.946129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:43.956198 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e8720-413c-48ba-b9dc-c36218fe6f34 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:44.012385 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d7b08-a9ca-4f3a-9256-6e4fc087b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:44.013532 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d7b08-a9ca-4f3a-9256-6e4fc087b0fb admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:44.013960 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d7b08-a9ca-4f3a-9256-6e4fc087b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:44.014248 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d7b08-a9ca-4f3a-9256-6e4fc087b0fb admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:44.015128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d7b08-a9ca-4f3a-9256-6e4fc087b0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:44.017565 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d7b08-a9ca-4f3a-9256-6e4fc087b0fb admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:44.089116 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1455/2898] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:01:43 2022] POST /identity/v3/users => generated 285 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:01:44.093308 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df400019-5dd5-4b3f-b364-b9f86afc0bc6 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:44.103738 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df400019-5dd5-4b3f-b364-b9f86afc0bc6 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:44.108549 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621c802-c150-49c0-bcdf-caadaab3babf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:44.109221 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621c802-c150-49c0-bcdf-caadaab3babf admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:44.109508 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621c802-c150-49c0-bcdf-caadaab3babf admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:44.109786 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621c802-c150-49c0-bcdf-caadaab3babf admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:44.110568 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621c802-c150-49c0-bcdf-caadaab3babf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:44.114439 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621c802-c150-49c0-bcdf-caadaab3babf admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:44.126745 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1444/2899] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:01:44.130948 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d7b08-a9ca-4f3a-9256-6e4fc087b0fb admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:44.141580 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d7b08-a9ca-4f3a-9256-6e4fc087b0fb admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:44.146102 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462e76b-62d0-45ec-849e-a82f159a1078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:44.146753 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462e76b-62d0-45ec-849e-a82f159a1078 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:44.147039 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462e76b-62d0-45ec-849e-a82f159a1078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:44.147355 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462e76b-62d0-45ec-849e-a82f159a1078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:44.148142 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462e76b-62d0-45ec-849e-a82f159a1078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:44.149727 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462e76b-62d0-45ec-849e-a82f159a1078 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:44.159314 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1456/2900] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:01:44.163156 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621c802-c150-49c0-bcdf-caadaab3babf admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:44.173134 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621c802-c150-49c0-bcdf-caadaab3babf admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:44.177720 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:44.178588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:44.179551 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:44.179807 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] PATH_INFO: `/v3/projects/80b694d41daa4f1e9285e606a5e67c0c/users/0733363f4f054494b91f4ac24f25ab22/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:44.193652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b694d41daa4f1e9285e606a5e67c0c, user_id=0733363f4f054494b91f4ac24f25ab22, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:44.196999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:44.266214 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1445/2901] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:01:44 2022] PUT /identity/v3/projects/80b694d41daa4f1e9285e606a5e67c0c/users/0733363f4f054494b91f4ac24f25ab22/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:01:44.277078 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffe59b-9530-4faf-aeec-20172f927b79 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:44.277374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffe59b-9530-4faf-aeec-20172f927b79 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:44.277661 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffe59b-9530-4faf-aeec-20172f927b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:44.304787 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-1dffe59b-9530-4faf-aeec-20172f927b79 None None] MFA Rules not processed for user `0733363f4f054494b91f4ac24f25ab22`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:01:44.305994 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-1dffe59b-9530-4faf-aeec-20172f927b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:44.367976 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1457/2902] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:01:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:01:44.379039 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:01:44.380726 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:01:44.436443 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6b410-5f91-49e0-b00a-68fb4b414456 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:01:44.440054 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3ab7b-6832-49d6-84c3-3742585e7753 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e939bc73c40140cb89517e918722d154', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:01:44.440690 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ab7b-6832-49d6-84c3-3742585e7753 service glance] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:01:44.440975 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ab7b-6832-49d6-84c3-3742585e7753 service glance] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:01:44.441250 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3ab7b-6832-49d6-84c3-3742585e7753 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:01:44.449760 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3ab7b-6832-49d6-84c3-3742585e7753 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:01:44.451168 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3ab7b-6832-49d6-84c3-3742585e7753 service glance] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:01:44.459825 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1446/2903] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:01:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:01:54.364547 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:01:54.455891 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:13.746664 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffe59b-9530-4faf-aeec-20172f927b79 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:13.805127 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffe59b-9530-4faf-aeec-20172f927b79 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:13.810306 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:13.819519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:13.819519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:13.819519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] PATH_INFO: `/v3/users/0733363f4f054494b91f4ac24f25ab22` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:13.819519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0733363f4f054494b91f4ac24f25ab22)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:13.819519 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:14.024335 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0733363f4f054494b91f4ac24f25ab22'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.024956 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0733363f4f054494b91f4ac24f25ab22'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.026065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0733363f4f054494b91f4ac24f25ab22'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.033308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0733363f4f054494b91f4ac24f25ab22'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.045685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0733363f4f054494b91f4ac24f25ab22'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.130739 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1458/2904] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:02:13 2022] DELETE /identity/v3/users/0733363f4f054494b91f4ac24f25ab22 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:02:14.135229 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaf423-37ef-451b-a961-3da23e5379da None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:14.135770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaf423-37ef-451b-a961-3da23e5379da None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:14.135968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceaf423-37ef-451b-a961-3da23e5379da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:14.159138 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-cceaf423-37ef-451b-a961-3da23e5379da None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:14.160367 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cceaf423-37ef-451b-a961-3da23e5379da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:14.217609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1447/2905] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:14.227241 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:14.229329 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:14.285293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d498ac-0847-4793-b0e0-d6132c0844b7 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:14.288729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd685a7-bdf1-4401-a5b4-0c802a1d8b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:14.289374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd685a7-bdf1-4401-a5b4-0c802a1d8b20 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:14.289657 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd685a7-bdf1-4401-a5b4-0c802a1d8b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:14.289933 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd685a7-bdf1-4401-a5b4-0c802a1d8b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:14.299448 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd685a7-bdf1-4401-a5b4-0c802a1d8b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:14.300855 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd685a7-bdf1-4401-a5b4-0c802a1d8b20 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:14.310202 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1459/2906] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:02:14.726588 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaf423-37ef-451b-a961-3da23e5379da None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:14.728109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-cceaf423-37ef-451b-a961-3da23e5379da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:14.740035 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceaf423-37ef-451b-a961-3da23e5379da None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:14.745445 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:14.746440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:14.746723 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:14.746999 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] PATH_INFO: `/v3/projects/80b694d41daa4f1e9285e606a5e67c0c` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:14.748258 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b694d41daa4f1e9285e606a5e67c0c)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:14.750182 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:14.846802 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b694d41daa4f1e9285e606a5e67c0c', 'request_id': 'req-83e87c54-6b80-44d7-a903-d94d7a60bbdc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.852166 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b694d41daa4f1e9285e606a5e67c0c', 'request_id': 'req-83e87c54-6b80-44d7-a903-d94d7a60bbdc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.915421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b694d41daa4f1e9285e606a5e67c0c', 'request_id': 'req-83e87c54-6b80-44d7-a903-d94d7a60bbdc'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.926740 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] The token cache is being invalidate because project 80b694d41daa4f1e9285e606a5e67c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:02:14.927029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.927714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-83e87c54-6b80-44d7-a903-d94d7a60bbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:14.929309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1448/2907] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:02:14 2022] DELETE /identity/v3/projects/80b694d41daa4f1e9285e606a5e67c0c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:02:14.934668 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48300742-db87-49ef-a4c2-2a64208b76f4 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:14.935176 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48300742-db87-49ef-a4c2-2a64208b76f4 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:14.935308 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48300742-db87-49ef-a4c2-2a64208b76f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:14.953161 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-48300742-db87-49ef-a4c2-2a64208b76f4 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:14.954349 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-48300742-db87-49ef-a4c2-2a64208b76f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:14.970670 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:15.013601 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1460/2908] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:15.021187 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636c3cd-fec5-4309-baad-52b1bdf3c1be None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.021474 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636c3cd-fec5-4309-baad-52b1bdf3c1be None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.021752 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636c3cd-fec5-4309-baad-52b1bdf3c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.040933 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-5636c3cd-fec5-4309-baad-52b1bdf3c1be None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:15.042126 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-5636c3cd-fec5-4309-baad-52b1bdf3c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:15.050288 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1449/2909] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:15.054571 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48300742-db87-49ef-a4c2-2a64208b76f4 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:15.064847 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48300742-db87-49ef-a4c2-2a64208b76f4 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:15.069515 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:15.070171 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.070460 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.070738 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] PATH_INFO: `/v3/domains` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.071496 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 12:02:15.071807 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:15.073961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:15.086173 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1461/2910] 10.222.0.26 () {58 vars in 1152 bytes} [Mon Jul 11 12:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 12:02:15.090437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636c3cd-fec5-4309-baad-52b1bdf3c1be None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:15.099548 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636c3cd-fec5-4309-baad-52b1bdf3c1be None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:15.104701 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92f677-6dc4-4e16-94b2-8be2c8755c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:15.105345 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92f677-6dc4-4e16-94b2-8be2c8755c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.105628 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92f677-6dc4-4e16-94b2-8be2c8755c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.105946 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92f677-6dc4-4e16-94b2-8be2c8755c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.106600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92f677-6dc4-4e16-94b2-8be2c8755c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:15.108507 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92f677-6dc4-4e16-94b2-8be2c8755c28 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:15.189459 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1450/2911] 10.222.0.26 () {60 vars in 1152 bytes} [Mon Jul 11 12:02:15 2022] POST /identity/v3/projects => generated 399 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 12:02:15.193478 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:15.203817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80483343-0e54-425c-8d64-e01fc3104824 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:15.258920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8c043-4c90-4167-8866-45561d992c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:15.259632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8c043-4c90-4167-8866-45561d992c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.259920 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8c043-4c90-4167-8866-45561d992c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.260204 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8c043-4c90-4167-8866-45561d992c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.260903 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8c043-4c90-4167-8866-45561d992c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:15.262817 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8c043-4c90-4167-8866-45561d992c05 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:15.350160 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1462/2912] 10.222.0.26 () {60 vars in 1143 bytes} [Mon Jul 11 12:02:15 2022] POST /identity/v3/users => generated 300 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 12:02:15.357239 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92f677-6dc4-4e16-94b2-8be2c8755c28 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:15.367421 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92f677-6dc4-4e16-94b2-8be2c8755c28 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:15.372621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f17dd-2bb0-4941-8442-6693eb501830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:15.373276 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f17dd-2bb0-4941-8442-6693eb501830 admin admin] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.373562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f17dd-2bb0-4941-8442-6693eb501830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.373839 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f17dd-2bb0-4941-8442-6693eb501830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.374566 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f17dd-2bb0-4941-8442-6693eb501830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:15.376175 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f17dd-2bb0-4941-8442-6693eb501830 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:15.386270 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1451/2913] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:02:15.389886 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8c043-4c90-4167-8866-45561d992c05 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:15.399927 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8c043-4c90-4167-8866-45561d992c05 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:15.404480 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7c9d-1c69-45b8-9e86-23ce0b848bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:15.405129 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7c9d-1c69-45b8-9e86-23ce0b848bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.405412 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7c9d-1c69-45b8-9e86-23ce0b848bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.405691 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7c9d-1c69-45b8-9e86-23ce0b848bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.406435 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd7c9d-1c69-45b8-9e86-23ce0b848bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:15.408050 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd7c9d-1c69-45b8-9e86-23ce0b848bc2 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:15.418705 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1463/2914] 10.222.0.26 () {58 vars in 1121 bytes} [Mon Jul 11 12:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 12:02:15.422546 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f17dd-2bb0-4941-8442-6693eb501830 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:15.432851 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f17dd-2bb0-4941-8442-6693eb501830 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:15.437424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:15.438294 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] REQUEST_METHOD: `PUT` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.438579 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.438856 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] PATH_INFO: `/v3/projects/feaa96785ddb4232bad4acdced14f9a0/users/3357b7ce4f544c6eb0b7b9ee28419f8e/roles/9d899c1b7b0740348cc8f312ed536187` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.453652 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] RBAC: Authorizing `identity:create_grant(project_id=feaa96785ddb4232bad4acdced14f9a0, user_id=3357b7ce4f544c6eb0b7b9ee28419f8e, role_id=9d899c1b7b0740348cc8f312ed536187)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:15.456304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:15.528062 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1452/2915] 10.222.0.26 () {60 vars in 1482 bytes} [Mon Jul 11 12:02:15 2022] PUT /identity/v3/projects/feaa96785ddb4232bad4acdced14f9a0/users/3357b7ce4f544c6eb0b7b9ee28419f8e/roles/9d899c1b7b0740348cc8f312ed536187 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:02:15.539059 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c6a05-a301-4332-acbf-9da9f2018420 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.539382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c6a05-a301-4332-acbf-9da9f2018420 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.539666 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c6a05-a301-4332-acbf-9da9f2018420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.566849 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-f23c6a05-a301-4332-acbf-9da9f2018420 None None] MFA Rules not processed for user `3357b7ce4f544c6eb0b7b9ee28419f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:15.568124 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f23c6a05-a301-4332-acbf-9da9f2018420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:15.631070 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1464/2916] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:02:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:15.642676 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:15.644382 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:15.666557 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:15.711598 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e9d2b-a4b8-4453-9ba4-35ffeed3e86d admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:15.715172 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca41381-8fd3-4e9f-8230-fd33ce636530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:15.715835 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca41381-8fd3-4e9f-8230-fd33ce636530 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:15.716118 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca41381-8fd3-4e9f-8230-fd33ce636530 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:15.716420 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca41381-8fd3-4e9f-8230-fd33ce636530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:15.724533 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca41381-8fd3-4e9f-8230-fd33ce636530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:15.725939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca41381-8fd3-4e9f-8230-fd33ce636530 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:15.735038 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1453/2917] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:02:24.514857 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c6a05-a301-4332-acbf-9da9f2018420 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:24.533195 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:24.576159 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c6a05-a301-4332-acbf-9da9f2018420 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:24.580729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:24.581583 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:24.581866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:24.582144 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] PATH_INFO: `/v3/users/3357b7ce4f544c6eb0b7b9ee28419f8e` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:24.583359 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3357b7ce4f544c6eb0b7b9ee28419f8e)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:24.585620 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:24.728866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3357b7ce4f544c6eb0b7b9ee28419f8e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:24.729514 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3357b7ce4f544c6eb0b7b9ee28419f8e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:24.730111 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3357b7ce4f544c6eb0b7b9ee28419f8e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:24.736740 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3357b7ce4f544c6eb0b7b9ee28419f8e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:24.748593 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3357b7ce4f544c6eb0b7b9ee28419f8e'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:24.795152 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1465/2918] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:02:24 2022] DELETE /identity/v3/users/3357b7ce4f544c6eb0b7b9ee28419f8e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:02:24.799621 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a58a8-33b1-4a75-b0dc-81c935f47ac1 None None] REQUEST_METHOD: `POST` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:24.800105 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a58a8-33b1-4a75-b0dc-81c935f47ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:24.800213 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a58a8-33b1-4a75-b0dc-81c935f47ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:24.817612 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.auth.core [None req-074a58a8-33b1-4a75-b0dc-81c935f47ac1 None None] MFA Rules not processed for user `41ca161a25324736922d4952cd76a1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:02:24.818815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-074a58a8-33b1-4a75-b0dc-81c935f47ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:24.876801 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1454/2919] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:02:24.886392 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:24.887842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:24.943481 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e1b2d-9d22-4d36-8122-1e86a1327db9 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:24.947133 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eee63d-1392-4c3f-8936-b37bb93d00f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:24.947812 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee63d-1392-4c3f-8936-b37bb93d00f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:24.948100 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee63d-1392-4c3f-8936-b37bb93d00f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:24.948382 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee63d-1392-4c3f-8936-b37bb93d00f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:24.957801 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eee63d-1392-4c3f-8936-b37bb93d00f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:24.959322 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eee63d-1392-4c3f-8936-b37bb93d00f5 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:24.968840 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1466/2920] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:02:25.423619 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a58a8-33b1-4a75-b0dc-81c935f47ac1 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:02:25.425058 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-074a58a8-33b1-4a75-b0dc-81c935f47ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:02:25.437685 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a58a8-33b1-4a75-b0dc-81c935f47ac1 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:02:25.442434 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:02:25.443354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:02:25.443669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:02:25.443908 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] PATH_INFO: `/v3/projects/feaa96785ddb4232bad4acdced14f9a0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:02:25.445220 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] RBAC: Authorizing `identity:delete_project(project_id=feaa96785ddb4232bad4acdced14f9a0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:02:25.447117 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:02:25.594553 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feaa96785ddb4232bad4acdced14f9a0', 'request_id': 'req-2764304f-e248-4128-8c70-1d96cce8593f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:25.600131 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feaa96785ddb4232bad4acdced14f9a0', 'request_id': 'req-2764304f-e248-4128-8c70-1d96cce8593f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:25.699336 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feaa96785ddb4232bad4acdced14f9a0', 'request_id': 'req-2764304f-e248-4128-8c70-1d96cce8593f'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:25.711053 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] The token cache is being invalidate because project feaa96785ddb4232bad4acdced14f9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:02:25.711378 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:25.712013 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2764304f-e248-4128-8c70-1d96cce8593f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:02:25.713768 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1455/2921] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:02:25 2022] DELETE /identity/v3/projects/feaa96785ddb4232bad4acdced14f9a0 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:02:34.964527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:02:35.708102 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:26.334469 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fa37e-5027-4aa3-9e76-f59d44c098bb None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:26.336954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fa37e-5027-4aa3-9e76-f59d44c098bb None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:26.336954 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fa37e-5027-4aa3-9e76-f59d44c098bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:26.353096 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-780fa37e-5027-4aa3-9e76-f59d44c098bb None None] MFA Rules not processed for user `fccd3de89cd94e5fa4c488c16d993241`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:03:26.354369 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-780fa37e-5027-4aa3-9e76-f59d44c098bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:26.408685 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1467/2922] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:03:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:03:27.386313 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732866-1562-48f1-b2c6-ce1a38542939 None None] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:27.386917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732866-1562-48f1-b2c6-ce1a38542939 None None] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:27.386917 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732866-1562-48f1-b2c6-ce1a38542939 None None] PATH_INFO: `/` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:27.388300 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1456/2923] 10.222.0.26 () {54 vars in 921 bytes} [Mon Jul 11 12:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 12:03:27.395841 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ac82e-3910-4747-9a64-cdd86c488e3d None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:27.396370 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ac82e-3910-4747-9a64-cdd86c488e3d None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:27.396544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ac82e-3910-4747-9a64-cdd86c488e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:27.448399 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-248ac82e-3910-4747-9a64-cdd86c488e3d None None] MFA Rules not processed for user `cb1fa1e11b1a44aaa31cc19f7c199285`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:03:27.449789 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-248ac82e-3910-4747-9a64-cdd86c488e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:27.527163 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1468/2924] 10.222.0.26 () {58 vars in 1018 bytes} [Mon Jul 11 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:03:27.541193 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732866-1562-48f1-b2c6-ce1a38542939 None None] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:27.543237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-ce732866-1562-48f1-b2c6-ce1a38542939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:27.558698 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732866-1562-48f1-b2c6-ce1a38542939 None None] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:27.562901 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:27.563757 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:27.564164 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:27.564545 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:27.573304 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:27.574865 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:27.587223 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1457/2925] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:03:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:03:27.684537 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ac82e-3910-4747-9a64-cdd86c488e3d None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:03:27.686292 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-248ac82e-3910-4747-9a64-cdd86c488e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:27.698861 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ac82e-3910-4747-9a64-cdd86c488e3d None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:03:27.702574 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:03:27.703223 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:03:27.703544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:03:27.703824 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:03:27.706061 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:03:27.777836 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:03:27.779992 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:03:27.790905 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1469/2926] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:03:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:03:37.581480 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=6 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:03:37.785871 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=92 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:04:58.146682 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:04:58.148230 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:04:58.168875 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84382300-0d39-4f03-ab51-e505e20eee10 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:04:58.174824 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:04:58.175605 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:04:58.175968 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:04:58.176321 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:04:58.178181 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:04:58.191967 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:04:58.193674 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:04:58.205522 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1458/2927] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:04:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:04:58.267038 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:04:58.278167 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed55ad-11ce-41ec-94d7-cbcb0e9981e4 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:04:58.282019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:04:58.282697 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:04:58.282987 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:04:58.283289 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:04:58.285232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:04:58.296233 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:04:58.297744 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:04:58.306792 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1470/2928] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:04:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:04:58.547527 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:04:58.556945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ac271-ac62-4a84-8792-3e9a938af2e2 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:04:58.560784 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:04:58.561429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:04:58.561714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:04:58.561992 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:04:58.563823 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:04:58.577399 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:04:58.578918 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:04:58.590734 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1459/2929] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:04:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:04:59.766966 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:04:59.775567 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fcc9b-3769-4a88-a403-6bef8f43d567 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:04:59.779101 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:04:59.779764 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:04:59.780048 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:04:59.780321 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:04:59.782115 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:04:59.795669 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:04:59.797077 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:04:59.808527 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1471/2930] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:04:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:02.312115 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:02.313594 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:02.328087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94388665-9eda-4ba8-9031-616f2d4e3685 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:02.334124 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:02.334783 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:02.335112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:02.335376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:02.337484 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:02.396357 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:02.397995 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:02.406945 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1460/2931] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:02.522135 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:02.531525 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6fc86-7ab9-4dc6-be3f-9e93bbb12ab3 service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:02.535080 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:02.535761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:02.536055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:02.536334 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:02.538088 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:02.602299 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:02.603703 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:02.613433 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1472/2932] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:09.435140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:09.435140 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:09.505424 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb9757-a4a3-4f63-a922-b7015c65797f service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:09.510354 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:09.510998 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:09.517464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:09.517464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:09.517464 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:09.524897 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:09.526479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:09.536274 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1461/2933] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:09.557323 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:09.558899 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:09.574035 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ff77a-2f6b-4652-b428-2c4d42bce6ee service neutron] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:09.578922 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:09.579842 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:09.580132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:09.580414 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] PATH_INFO: `/v3/users/fa171a9679a3460bb2ee9369e4051690` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:09.581608 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa171a9679a3460bb2ee9369e4051690)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:09.583909 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:09.757449 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa171a9679a3460bb2ee9369e4051690'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:09.758025 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa171a9679a3460bb2ee9369e4051690'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:09.758881 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa171a9679a3460bb2ee9369e4051690'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:09.766064 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa171a9679a3460bb2ee9369e4051690'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:09.778438 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa171a9679a3460bb2ee9369e4051690'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:09.858240 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1473/2934] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:05:09 2022] DELETE /identity/v3/users/fa171a9679a3460bb2ee9369e4051690 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:05:10.081391 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:10.094112 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f246e-a046-457b-9efe-8b2a79ff5c63 service placement] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:10.147589 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:10.148435 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:10.148714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:10.148982 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] PATH_INFO: `/v3/projects/59ae388052ee406c8d6cc749a1ed8c02` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:10.150815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ae388052ee406c8d6cc749a1ed8c02)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:10.152714 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:10.155639 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=100 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:10.231930 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ae388052ee406c8d6cc749a1ed8c02', 'request_id': 'req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.237808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ae388052ee406c8d6cc749a1ed8c02', 'request_id': 'req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.293185 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ae388052ee406c8d6cc749a1ed8c02', 'request_id': 'req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.306064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] The token cache is being invalidate because project 59ae388052ee406c8d6cc749a1ed8c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:05:10.306393 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.307032 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.308634 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1462/2935] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:05:10 2022] DELETE /identity/v3/projects/59ae388052ee406c8d6cc749a1ed8c02 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:05:10.312600 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:10.314055 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:10.334042 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748cc43-a64c-4ade-ab3b-6098f1a1f24c admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:10.372640 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:10.391076 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:10.391980 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:10.392263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:10.392538 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] PATH_INFO: `/v3/users/dcaf550f17164636be11f4400fc9c753` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:10.393729 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcaf550f17164636be11f4400fc9c753)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:10.395720 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:10.539877 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcaf550f17164636be11f4400fc9c753'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.540545 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcaf550f17164636be11f4400fc9c753'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.541164 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcaf550f17164636be11f4400fc9c753'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.548019 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcaf550f17164636be11f4400fc9c753'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.560131 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcaf550f17164636be11f4400fc9c753'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:10.646300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1474/2936] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:05:10 2022] DELETE /identity/v3/users/dcaf550f17164636be11f4400fc9c753 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:05:10.890029 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:10.892113 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:10.905108 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee65d7-8c26-4e2c-a6ff-ba79ffa21344 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:10.967429 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:10.968291 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:10.968574 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:10.968852 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] PATH_INFO: `/v3/projects/2010c56213b144dfac669fe9c4ff5ae0` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:10.970048 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2010c56213b144dfac669fe9c4ff5ae0)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:10.971986 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:11.076012 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2010c56213b144dfac669fe9c4ff5ae0', 'request_id': 'req-2c59a978-3f02-4992-a81f-03c2bd25a736'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:11.081372 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2010c56213b144dfac669fe9c4ff5ae0', 'request_id': 'req-2c59a978-3f02-4992-a81f-03c2bd25a736'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:11.120904 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2010c56213b144dfac669fe9c4ff5ae0', 'request_id': 'req-2c59a978-3f02-4992-a81f-03c2bd25a736'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:11.131815 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] The token cache is being invalidate because project 2010c56213b144dfac669fe9c4ff5ae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:05:11.132106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:11.132719 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:05:11.134236 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1463/2937] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:05:10 2022] DELETE /identity/v3/projects/2010c56213b144dfac669fe9c4ff5ae0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:05:20.642622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:21.128796 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=74,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:22.129053 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:22.130445 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:22.226230 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295372d9-7b4d-4b10-ae47-a72b3fc624ec admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:22.282134 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2045dfc-de10-404c-9f35-977616bcd25a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:22.282776 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2045dfc-de10-404c-9f35-977616bcd25a service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:22.283058 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2045dfc-de10-404c-9f35-977616bcd25a service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:22.283364 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2045dfc-de10-404c-9f35-977616bcd25a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:22.285354 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-f2045dfc-de10-404c-9f35-977616bcd25a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:22.296146 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2045dfc-de10-404c-9f35-977616bcd25a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:22.297696 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2045dfc-de10-404c-9f35-977616bcd25a service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:22.307845 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1475/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:32.303122 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=71 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:33.457020 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:33.458431 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:33.515343 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59a978-3f02-4992-a81f-03c2bd25a736 admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:33.518854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7cebda9ae4e4a6b9a9bbface3e7b232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:33.519526 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:33.519808 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:33.520084 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:33.521854 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:33.577669 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:33.579106 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:33.588237 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1464/2939] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:34.740731 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1b906-5ad7-4715-b45f-51a1b42b5ec2 None None] REQUEST_METHOD: `POST` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:34.741214 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1b906-5ad7-4715-b45f-51a1b42b5ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:34.741325 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1b906-5ad7-4715-b45f-51a1b42b5ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:34.758344 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.auth.core [None req-57d1b906-5ad7-4715-b45f-51a1b42b5ec2 None None] MFA Rules not processed for user `fccd3de89cd94e5fa4c488c16d993241`. Rule list: `[]` (Enabled: `True`). {{(pid=80500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 12:05:34.759625 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-57d1b906-5ad7-4715-b45f-51a1b42b5ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:34.767374 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1476/2940] 10.222.0.26 () {58 vars in 957 bytes} [Mon Jul 11 12:05:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 12:05:35.085109 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:35.086496 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:35.141376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8aad01-e48a-4c72-9027-33c771476462 service cinder] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:35.144813 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:35.145447 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:35.145728 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:35.146005 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:35.147736 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:35.156873 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:35.158290 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:35.166351 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1465/2941] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:35.825866 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1b906-5ad7-4715-b45f-51a1b42b5ec2 None None] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:35.827309 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-57d1b906-5ad7-4715-b45f-51a1b42b5ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:35.842872 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1b906-5ad7-4715-b45f-51a1b42b5ec2 None None] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:35.848244 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:35.848896 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:35.849232 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:35.849513 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:35.851305 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:35.864253 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:35.865622 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:35.875870 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1477/2942] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:05:45.162562 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=74 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:45.871523 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=9 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:05:50.367571 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:05:50.369200 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:50.440939 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1f492-306e-4622-b084-683c87983bd5 service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:05:50.446437 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:05:50.447076 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:05:50.447386 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:05:50.447664 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:05:50.449440 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:05:50.464635 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:05:50.466251 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:05:50.476879 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1466/2943] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:06:00.472323 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=34 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:42.466596 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:42.468822 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:42.538756 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba2b0e-9bb9-4dc2-b8e5-b7cfceb9d739 service placement] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:42.544998 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '258ce5f4d6ff400199a80157cc34f5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:42.545187 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:42.545544 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:42.545832 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:42.547641 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:42.560630 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:42.562281 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:42.574936 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1478/2944] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:49.434408 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:49.471479 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d119ca-1720-4880-88d2-c8c9346b3e69 service nova] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:49.475636 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb1fa1e11b1a44aaa31cc19f7c199285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:49.476271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] REQUEST_METHOD: `GET` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:49.476554 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:49.476830 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:49.478609 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:49.538271 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:49.539732 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:49.549056 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1467/2945] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:07:52.572632 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=22 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:56.347682 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:56.349113 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:56.363761 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45d5c9-9127-40b6-9618-ad9c5ea3a799 service nova] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:56.368189 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:56.371550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:56.371550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:56.371550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] PATH_INFO: `/v3/users/fccd3de89cd94e5fa4c488c16d993241` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:56.371550 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fccd3de89cd94e5fa4c488c16d993241)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:56.372901 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:56.547843 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fccd3de89cd94e5fa4c488c16d993241'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:56.548489 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fccd3de89cd94e5fa4c488c16d993241'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:56.549094 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fccd3de89cd94e5fa4c488c16d993241'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:56.555953 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fccd3de89cd94e5fa4c488c16d993241'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:56.567365 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fccd3de89cd94e5fa4c488c16d993241'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:56.649263 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1479/2946] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:07:56 2022] DELETE /identity/v3/users/fccd3de89cd94e5fa4c488c16d993241 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 12:07:56.896010 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:56.905844 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afb537-aa8f-4c8a-96dc-92ead4946f5e service neutron] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:56.963491 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:56.964349 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:56.964632 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:56.964905 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] PATH_INFO: `/v3/projects/79c8c84b3f704d528ea772607da56ddf` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:56.966104 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c8c84b3f704d528ea772607da56ddf)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:56.968064 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:57.080268 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c8c84b3f704d528ea772607da56ddf', 'request_id': 'req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.086718 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c8c84b3f704d528ea772607da56ddf', 'request_id': 'req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.174788 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c8c84b3f704d528ea772607da56ddf', 'request_id': 'req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.182600 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:57.187087 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] The token cache is being invalidate because project 79c8c84b3f704d528ea772607da56ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:07:57.187515 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.188161 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.189749 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1468/2947] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:07:56 2022] DELETE /identity/v3/projects/79c8c84b3f704d528ea772607da56ddf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:07:57.194288 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:57.195716 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:57.209443 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc88ebf-9afc-428f-a484-7c934f4a4a09 admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:57.269128 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:57.270016 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:57.270316 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:57.270575 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] PATH_INFO: `/v3/users/7209127017464bb48e2d45e168ebeacf` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:57.272398 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7209127017464bb48e2d45e168ebeacf)` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:57.274300 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:57.475293 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7209127017464bb48e2d45e168ebeacf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.475939 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7209127017464bb48e2d45e168ebeacf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.476455 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7209127017464bb48e2d45e168ebeacf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.483216 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7209127017464bb48e2d45e168ebeacf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.495394 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.notifications [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7209127017464bb48e2d45e168ebeacf'} {{(pid=80500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:57.500617 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:07:57.580304 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1480/2948] 10.222.0.26 () {58 vars in 1223 bytes} [Mon Jul 11 12:07:57 2022] DELETE /identity/v3/users/7209127017464bb48e2d45e168ebeacf => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:07:57.858297 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] Authenticating user token {{(pid=80499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:07:57.859769 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.fernet_utils [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:07:57.875312 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a9a341-f8f5-4aac-ab71-bcf0433be67b admin admin] Validating token access rules against request {{(pid=80499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:07:57.930770 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41ca161a25324736922d4952cd76a1f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd16c61acfee74652b6cd4c256cd8f443', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:07:57.931654 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:07:57.931936 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] SCRIPT_NAME: `/identity` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:07:57.932209 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] PATH_INFO: `/v3/projects/727f4d20c6ff48c5befe4ebab07a3c67` {{(pid=80499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:07:57.933376 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] RBAC: Authorizing `identity:delete_project(project_id=727f4d20c6ff48c5befe4ebab07a3c67)` {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:07:57.935266 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] RBAC: Authorization granted {{(pid=80499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:07:58.062573 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727f4d20c6ff48c5befe4ebab07a3c67', 'request_id': 'req-514991bb-7d4f-46aa-9574-d508addd423c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:58.068027 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727f4d20c6ff48c5befe4ebab07a3c67', 'request_id': 'req-514991bb-7d4f-46aa-9574-d508addd423c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:58.176774 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727f4d20c6ff48c5befe4ebab07a3c67', 'request_id': 'req-514991bb-7d4f-46aa-9574-d508addd423c'} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:58.189019 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] The token cache is being invalidate because project 727f4d20c6ff48c5befe4ebab07a3c67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 12:07:58.189309 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:58.189929 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG keystone.notifications [None req-514991bb-7d4f-46aa-9574-d508addd423c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 12:07:58.191569 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: [pid: 80499|app: 0|req: 1469/2949] 10.222.0.26 () {58 vars in 1232 bytes} [Mon Jul 11 12:07:57 2022] DELETE /identity/v3/projects/727f4d20c6ff48c5befe4ebab07a3c67 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 12:08:07.696695 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:08:08.185920 dv-cinder-848429-2-iscsi devstack@keystone.service[80499]: DEBUG dbcounter [-] [80499] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 12:10:25.167724 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] Authenticating user token {{(pid=80500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 12:10:25.169132 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:10:25.258109 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691b66a-0292-4d6a-b61f-f8d66c09185a admin admin] Validating token access rules against request {{(pid=80500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 12:10:25.314156 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c7f7d-7be9-4a39-99e8-88a9fc01652c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f2124d6d4e46ca9baa9728741b0e4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a099df6dc96f4edeb762df2a36068788', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 12:10:25.314733 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7f7d-7be9-4a39-99e8-88a9fc01652c service placement] REQUEST_METHOD: `GET` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 12:10:25.315023 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7f7d-7be9-4a39-99e8-88a9fc01652c service placement] SCRIPT_NAME: `/identity` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 12:10:25.315326 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c7f7d-7be9-4a39-99e8-88a9fc01652c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 12:10:25.317065 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.fernet_utils [None req-7d1c7f7d-7be9-4a39-99e8-88a9fc01652c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 12:10:25.328419 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c7f7d-7be9-4a39-99e8-88a9fc01652c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 12:10:25.329961 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c7f7d-7be9-4a39-99e8-88a9fc01652c service placement] RBAC: Authorization granted {{(pid=80500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 12:10:25.339662 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: [pid: 80500|app: 0|req: 1481/2950] 10.222.0.26 () {60 vars in 1365 bytes} [Mon Jul 11 12:10:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 12:10:35.335474 dv-cinder-848429-2-iscsi devstack@keystone.service[80500]: DEBUG dbcounter [-] [80500] Writing DB stats keystone:SELECT=67 {{(pid=80500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}